AGV robot operation control method and system

Abstract

The invention discloses an AGV robot operation control method and system, and relates to the technical field of AGV robots, and the method comprises the steps: collecting environment data based on a sensor, obtaining state data through an AGV robot, and building an initial path based on map data through an A * algorithm; a grey wolf optimization algorithm is used to generate candidate paths, and the paths are adjusted for position updating to output an optimal path; aGV robot state vectors are constructed, and AGV robot actions are set. According to the method, the initial path is generated through the A * algorithm, the multiple candidate paths are generated through the grey wolf optimization algorithm on the basis, diversified path selection is formed, the path planning flexibility is improved, the path can be globally searched and locally adjusted through grey wolf optimization on the basis of the initial path by jointly using the two paths, and the path planning efficiency is improved. The method is advantaged in that local optimization of the path is avoided, global optimality of path planning is improved, and through combined use of the grey wolf optimization A algorithm and the DQN model, the AGV robot can realize real-time response and autonomous learning in a complex dynamic environment.

Description

Technical Field

[0001] The present invention relates to the technical field of AGV robots, and in particular to an AGV robot operation control method and system. Background Art

[0002] The AGV robot system is a production system based on AGV robots that carry out material handling and item sorting through preset paths. The use of AGV robots effectively improves the efficiency and accuracy of production and logistics processes. In the existing AGV robot control method, a pre-set fixed path or simple rules are usually used to achieve path planning and obstacle avoidance;

[0003] The existing AGV robot control methods have good effects in searching for the global optimal path, but they are prone to fall into local optimal solutions in dynamic environments and lack global optimization capabilities. At the same time, swarm intelligence optimization methods can improve the optimality of path planning through global search, but these methods have large computational complexity and slow convergence speed when dealing with multi-dimensional complex path optimization, especially in scenes with dense obstacles or frequent environmental changes. The problem of inefficient path planning is prone to occur, resulting in a decrease in the path movement efficiency during the operation of the AGV robot, affecting the efficiency of the overall production or logistics process. Summary of the invention

[0004] In view of the problems existing in the above-mentioned existing AGV robot operation control method and system, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that the existing AGV robot control method has a good effect in searching for the global optimal path, but it is easy to fall into the local optimal solution in a dynamic environment and lacks global optimization capability. At the same time, the swarm intelligence optimization method can improve the optimality of path planning through global search, but these methods have a large amount of calculation and slow convergence speed when dealing with multi-dimensional complex path optimization, especially in scenes with dense obstacles or frequent environmental changes. The problem of low path planning efficiency is prone to occur, resulting in a decrease in path movement efficiency during the operation of the AGV robot, affecting the efficiency of the overall production or logistics process.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an AGV robot operation control method, which comprises:

[0007] Collect environmental data based on sensors, obtain status data through AGV robots, and use A based on map data * The algorithm constructs the initial path;

[0008] Use the Gray Wolf Optimization Algorithm to generate candidate paths, adjust the paths to update the positions and output the optimal paths;

[0009] Construct the AGV robot state vector and set the AGV robot action, update and iterate the Q value based on the DQN model through the Bellman equation, and train the DQN model;

[0010] The DQN model is tested in scenarios and deployed to the AGV robot. The collection, transmission and storage of AGV robot data are securely encrypted and the AGV robot data is backed up.

[0011] As a preferred solution of the AGV robot operation control method of the present invention, wherein: the environmental data is collected based on the sensor, and the status data is obtained through the AGV robot, including:

[0012] Obtain static and dynamic data of the environment where the AGV robot is located through lidar, camera and ultrasonic sensor;

[0013] The static data includes the position data of fixed obstacles in the environment acquired by the laser radar sensor and constructing two-dimensional grid map data, and marking the coordinates of the fixed obstacles in the map data;

[0014] Dynamic data includes real-time updates of the relative positions and velocities of dynamic objects in the environment detected by cameras and lidar;

[0015] Based on the sensors inside the AGV robot, the robot obtains its own status data, including the current coordinate position, speed, acceleration and direction angle;

[0016] Define the starting point coordinates and target point coordinates of the AGV robot and represent them in the grid map data.

[0017] As a preferred solution of the AGV robot operation control method of the present invention, wherein: the map data is based on A * The algorithm constructs the initial path, including,

[0018] Based on the two-dimensional grid map data, the two-dimensional grid map data is marked as passable or impassable according to the coordinates of fixed obstacles and the coordinates of dynamic objects in the detection environment;

[0019] Use A * The algorithm calculates the Euclidean distance h(x,y) based on the current position coordinates of the AGV robot and the target point coordinates;

[0020] Based on the starting point coordinates of the AGV robot, define the moving distance from the starting point coordinates to the current node, and express the distance from the current position coordinates of the AGV robot to the starting point coordinates as g(x,y);

[0021] Calculate the sum of h(x,y) and g(x,y) to represent A *The comprehensive cost of the algorithm f(x,y);

[0022] Based on A * The algorithm creates an open list and a closed list. The open list stores the set of nodes to be expanded. Each node records the current coordinates, the current g(x,y) value, h(x,y) value and f(x,y) comprehensive cost value. The closed list records the coordinates of nodes that have been expanded.

[0023] Select the node coordinate with the smallest comprehensive cost f(x,y) as the current position coordinate C, and move the node coordinate C from the open list to the closed list;

[0024] If the current position coordinate C is not the target point coordinate, continue to expand the adjacent nodes, check the four adjacent nodes of the current position coordinate C, calculate the adjacent node g(x,y) value, h(x,y) value and f(x,y) comprehensive cost value, and move the expanded node coordinates to the closed list;

[0025] If the current position coordinate C is the target point coordinate, then the node before the node extension is traced until it is traced to the starting point coordinate, and the node coordinates in the tracing process are marked as the initial path of the AGV robot.

[0026] As a preferred solution of the AGV robot operation control method of the present invention, wherein: the use of the gray wolf optimization algorithm to generate candidate paths, adjust the paths to update the positions and output the optimal paths includes:

[0027] Use the gray wolf optimization algorithm to set the number of wolves n, select the first n-1 node coordinates greater than the minimum value of the comprehensive cost f(x, y) according to the number of wolves n-1 as the current position coordinates D, and repeat the expansion of adjacent nodes to form a candidate path for the AGV robot, and combine the candidate path and the initial path of the AGV robot into a candidate solution set;

[0028] Based on the total path length and the number of obstacles passed by the candidate solution path, the fitness of each candidate solution is calculated, which is expressed as:

[0029]

[0030] X={(x1,y1),(x2,y2),Λ,(x ε ,y ε )};

[0031] In the formula, f(X i ) represents the fitness value of the ith path in the candidate solution set, L(X i ) represents the total length of the i-th path, C(X i ) represents the number of obstacles passed by the i-th path, X represents the vector of the path, including the coordinates of all nodes of the path, xε and ε Respectively represent the x-coordinate and y-coordinate of the ε-th node of path X;

[0032] The fitness of different paths is calculated and sorted, and the top three paths are selected as reference paths, and the remaining paths are selected as adjustment paths for position update, which is expressed as:

[0033]

[0034] D(X α ,X i )={(d 1,x ,d 1,y ),(d 2,x ,d 2,y ),Λ,(d ε,x ,d ε,y )};

[0035] Where, X i (t+1) represents the i-th path with iteration number t+1, X α (t), X β (t) and X δ (t) represents the optimal path α, the second-best path β and the third-best path δ of iteration number t, A represents the adjustment coefficient based on the number of iterations, D(Xα,Xi) represents the difference direction of each node coordinate between the optimal path α and the i-th path, T represents the maximum number of iterations, d ε,x and d ε,y Respectively represent the path X i and path X α The difference between the x-coordinate and the y-coordinate of the εth node;

[0036] The fitness value of the path is recalculated in each iteration, and the position of the adjusted path is updated based on the paths with the best, second best and third best fitness values ​​of the current iteration as reference paths;

[0037] Based on the changes of the optimal, suboptimal and third-optimal fitness of the current iteration t and the previous iteration t-1, the change rates are calculated respectively, and the average value of the optimal, suboptimal and third-optimal fitness change rates is calculated as the comprehensive change rate, the comprehensive change rate of the previous s iterations is counted, and the average value of the comprehensive change rate of the previous s iterations is calculated as the adaptive threshold;

[0038] If the comprehensive change rate after each iteration is greater than or equal to the adaptive threshold, the iteration is stopped and the optimal fitness path of the current iteration is output as the optimal path.

[0039] As a preferred solution of the AGV robot operation control method of the present invention, wherein: the AGV robot state vector is constructed and the AGV robot action is set, the Q value is updated and iteratively calculated based on the DQN model through the Bellman equation, and the DQN model is trained, including:

[0040] Based on the gray wolf optimization algorithm, the optimal fitness path is used as the optimal path, and the state vector S is constructed through the state data of the AGV robot. t , set the AGV robot actions to include forward θ1, left turn θ2, right turn θ3, deceleration θ4 and stop θ5, and define the immediate reward r of the action w Including making the AGV robot approach the target point coordinates, avoid obstacles, collide with obstacles and reach the target point coordinates;

[0041] Based on the DQN model, the Q value of each possible action is calculated, and the action with the largest Q value is selected for execution, which is expressed as:

[0042]

[0043] In the formula, Q(s w ,θ w ) represents the Q value of executing action θ in state s at the current time w, r w represents the immediate reward of the action at time w, represents the discount factor for time w, Q(s w+1 ,θ' w+1 ) represents any action θ' in the state s at time w+1 w+1 The corresponding Q value is, represents the maximum discount factor, d w Indicates the distance between the AGV robot and the target point coordinates at time w, d max Indicates the distance from the starting point coordinates to the target point coordinates of the AGV robot;

[0044] The Q value is calculated through the Bellman equation. During the iterative update process, the current state vector, action, reward, and next state vector are recorded and stored in the experience replay pool.

[0045] Based on the stored data in the experience replay pool as training data, the DQN network is trained through the mean square error, and the model parameters are iteratively optimized through the Adam optimizer and the gradient descent method. When the calculated error value no longer decreases significantly during the continuous iteration process, the iteration is stopped and the model parameters are output to update the DQN model.

[0046] As a preferred solution of the AGV robot operation control method of the present invention, wherein: the scenario testing of the DQN model and deployment to the AGV robot includes:

[0047] The trained DQN model is deployed in a simulation environment for scenario testing. The sensor parameters are modified through human intervention to detect and identify the execution actions of the AGV robot. At the same time, the environmental parameters of the simulation environment, including temperature, humidity and light intensity, are adjusted to test the execution actions of the AGV robot in different environments. Safety strategies are set for the AGV robot, including braking and parking, and the DQN model is deployed to the AGV robot.

[0048] As a preferred solution of the AGV robot operation control method of the present invention, wherein: the collection, transmission and storage of AGV robot data are securely encrypted and transmitted, and the AGV robot data is backed up, including:

[0049] Use AES encryption interface to collect data from AGV robots, and adopt timestamp mechanism to record the time information of data generation and transmission;

[0050] The TLS protocol is used to encrypt data transmission, and the transmitted data is symmetrically encrypted using AES-256, and backed up and stored in the cloud server and storage hard disk.

[0051] Another object of the present invention is to provide a system for an AGV robot operation control method, which comprises:

[0052] The data acquisition module collects static and dynamic data of the environment where the AGV robot is located, and obtains the status data of the AGV robot;

[0053] The path generation module uses A * The algorithm constructs an initial path and determines the global path from the starting point to the target point;

[0054] The path optimization module uses the gray wolf optimization algorithm to generate multiple candidate paths based on the initial path, and selects the optimal path through the position update strategy;

[0055] The action strategy module uses the DQN model to train the action strategy of the AGV robot, updates the Q value through the Bellman equation, and updates the model parameters through the mean square error and Adam optimizer;

[0056] The scenario testing module deploys the trained DQN model to the AGV robot and performs scenario testing in a simulated environment;

[0057] The data security encryption module performs secure encryption processing on the data collection, transmission and storage of the AGV robot.

[0058] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned AGV robot operation control method when executing the computer program.

[0059] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned AGV robot operation control method when executed by a processor.

[0060] The beneficial effects of the present invention are: * The algorithm generates an initial path, and on this basis, uses the Gray Wolf Optimization Algorithm to generate multiple candidate paths, forming a variety of path choices and improving the flexibility of path planning. The combined use of the two can be based on the preliminary path. Through the Gray Wolf Optimization, the path can be globally searched and locally adjusted to avoid the path falling into the local optimum, thereby improving the global optimality of path planning. Through the combined use of the Gray Wolf Optimization Algorithm and the DQN model, the AGV robot can achieve real-time response and autonomous learning in a complex dynamic environment. The Gray Wolf Optimization Algorithm dynamically adjusts the path globally, while the DQN model locally realizes intelligent obstacle avoidance and path fine-tuning, enabling the AGV robot to flexibly respond to various changes in the environment and improve the real-time and autonomous learning capabilities of path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0062] Figure 1 It is a flowchart of the AGV robot operation control method.

[0063] Figure 2 This is a schematic diagram of the structure of the AGV robot operation control system. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0066] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0067] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and which provides a system for an AGV robot operation control method. The system for the AGV robot operation control method includes:

[0068] S1, collects environmental data based on sensors, obtains status data through AGV robots, and uses A based on map data * The algorithm constructs the initial path;

[0069] Preferably, the environmental data is collected based on the sensor, and the status data is obtained by the AGV robot, including:

[0070] Obtain static and dynamic data of the environment where the AGV robot is located through lidar, camera and ultrasonic sensor;

[0071] The static data includes the position data of fixed obstacles in the environment acquired by the laser radar sensor and constructing two-dimensional grid map data, and marking the coordinates of the fixed obstacles in the map data;

[0072] Dynamic data includes real-time updates of the relative positions and velocities of dynamic objects in the environment detected by cameras and lidar;

[0073] Based on the sensors inside the AGV robot, such as the IMU inertial measurement unit, encoder, etc., it obtains its own status data, including the current coordinate position, speed, acceleration and direction angle;

[0074] Define the starting point coordinates and target point coordinates of the AGV robot and represent them in the grid map data.

[0075] By obtaining the information of fixed obstacles in the static environment through the laser radar and accurately representing it in the two-dimensional grid map, a clear environmental model can be formed, in which the coordinate position of the static obstacle is fixed and accurately marked, so that the AGV robot can predict the position of the obstacle when planning the path, thereby accurately calculating the optimal path and improving the reliability of path planning. By combining the camera and the laser radar to detect dynamic objects in the environment in real time, such as pedestrians and mobile equipment, the relative position and speed information of these objects can be updated in real time. The AGV robot's own state data, such as the current position, speed, acceleration and direction angle, can be obtained through internal sensors such as the IMU inertial measurement unit and encoder, which can accurately track the real-time position and posture of the AGV robot. By marking the starting point coordinates and the target point coordinates in the grid map, global path planning is performed based on the static obstacle data to ensure that the overall path of the AGV robot from the starting point to the target point is optimal. The global planning based on the grid map combined with the local adjustment of dynamic obstacle avoidance can ensure that the AGV robot can achieve autonomous navigation in a complex environment. By obtaining environmental data and its own state data in real time, the AGV robot can autonomously identify obstacles, avoid risks, and dynamically adjust the path, which significantly improves the autonomous navigation capability.

[0076] Further, based on the map data, A * The algorithm constructs the initial path, including,

[0077] Based on the two-dimensional grid map data, the two-dimensional grid map data is marked as passable or impassable according to the coordinates of fixed obstacles and the coordinates of dynamic objects in the detection environment;

[0078] Use A * The algorithm calculates the Euclidean distance based on the current position coordinates of the AGV robot and the target point coordinates, which is expressed as:

[0079]

[0080] In the formula, h(x,y) represents the Euclidean distance between the target point coordinates and the current position coordinates, x G and G Respectively represent the x-coordinate and y-coordinate of the target point, and x and y represent the x-coordinate and y-coordinate of the current position of the AGV robot;

[0081] Based on the starting point coordinates of the AGV robot, define the moving distance from the starting point coordinates to the current node, and express the distance from the current position coordinates of the AGV robot to the starting point coordinates as g(x,y);

[0082] Calculate the sum of h(x,y) and g(x,y) to represent A * The comprehensive cost of the algorithm f(x,y);

[0083] Based on A * The algorithm creates an open list and a closed list. The open list stores the set of nodes to be expanded. Each node records the current coordinates, the current g(x,y) value, h(x,y) value and f(x,y) comprehensive cost value. The closed list records the coordinates of nodes that have been expanded.

[0084] Select the node coordinate with the smallest comprehensive cost f(x,y) as the current position coordinate C, and move the node coordinate C from the open list to the closed list;

[0085] If the current position coordinate C is not the target point coordinate, continue to expand the adjacent nodes, check the four adjacent nodes of the current position coordinate C, calculate the adjacent node g(x,y) value, h(x,y) value and f(x,y) comprehensive cost value, and move the expanded node coordinates to the closed list;

[0086] If the current position coordinate C is the target point coordinate, then the node before the node extension is traced until it is traced to the starting point coordinate, and the node coordinates in the tracing process are marked as the initial path of the AGV robot.

[0087] By marking the passable and impassable areas on the two-dimensional grid map, fixed obstacles and dynamic obstacles can be effectively avoided. The Euclidean distance is used to calculate the distance between the current position of the AGV robot and the target point. As one of the reference indicators for path planning, it can help select the shortest path and improve the accuracy and reliability of path planning. By dynamically updating the obstacle coordinates on the grid map and combining the real-time detected dynamic object coordinates with the grid map, it can quickly respond to environmental changes and re-mark passable and impassable areas. When expanding nodes, the nodes with the lowest comprehensive cost are given priority (that is, considering the current distance to the starting point and the current distance to the target). It can quickly find the optimal path in a dynamic environment and improve the flexibility of path planning.

[0088] The Euclidean distance between the current position of the AGV robot and the target point and the moving distance from the starting point to the current position are combined through the comprehensive cost function, which can not only ensure the optimality of the path, but also avoid falling into the local optimal solution. * The algorithm usually only considers the cost of moving from the starting point to the current position. In this scheme, the estimated cost from the current node to the target point and the cost of moving from the starting point to the current position are comprehensively considered. By comprehensively considering the historical cost and the future estimated cost, it can avoid wandering in the local optimal solution, effectively improving the success rate and efficiency of path planning in complex environments;

[0089] Using open lists and closed lists to manage nodes to be expanded and nodes that have been expanded can effectively reduce repeated calculations and improve the efficiency of path planning. The open list records all nodes currently to be expanded, and always selects the nodes with the smallest comprehensive cost for expansion to ensure the optimality and efficiency of path planning. The closed list records the nodes that have been expanded, avoiding repeated calculations of nodes that have been processed, improving the execution efficiency of the algorithm, and effectively reducing the amount of calculation in complex environments. By tracing back the expanded nodes to form the initial path of the AGV robot, the process and results of path planning can be clearly presented, improving the interpretability of path planning.

[0090] S2, use the gray wolf optimization algorithm to generate candidate paths, adjust the paths to update the positions and output the optimal paths;

[0091] Preferably, the gray wolf optimization algorithm is used to generate candidate paths, and the paths are adjusted to update the positions and output the optimal paths, including:

[0092] Use the gray wolf optimization algorithm to set the number of wolves n, select the first n-1 node coordinates greater than the minimum value of the comprehensive cost f(x, y) according to the number of wolves n-1 as the current position coordinates D, and repeat the expansion of adjacent nodes to form a candidate path for the AGV robot, and combine the candidate path and the initial path of the AGV robot into a candidate solution set;

[0093] Based on the total path length and the number of obstacles passed by the candidate solution path, the fitness of each candidate solution is calculated, which is expressed as:

[0094]

[0095] X={(x1,y1),(x2,y2),Λ,(x ε ,y ε )};

[0096] In the formula, f(X i ) represents the fitness value of the ith path in the candidate solution set, L(X i ) represents the total length of the i-th path, C(X i ) represents the number of obstacles passed by the i-th path, X represents the vector of the path, including the coordinates of all nodes of the path, x ε and ε Respectively represent the x-coordinate and y-coordinate of the ε-th node of path X;

[0097] The fitness of different paths is calculated and sorted, and the top three paths are selected as reference paths, and the remaining paths are selected as adjustment paths for position update, which is expressed as:

[0098]

[0099] D(X α ,X i )={(d 1,x ,d 1,y ),(d 2,x ,d 2,y ),Λ,(d ε,x ,d ε,y )};

[0100] Where, X i (t+1) represents the i-th path with iteration number t+1, X α (t), X β (t) and X δ (t) represent the optimal path α, the second-best path β and the third-best path δ of iteration number t, A represents the adjustment coefficient based on the iteration number, D(X α ,X i ) represents the difference direction of the coordinates of each node between the optimal path α and the i-th path, T represents the maximum number of iterations, d ε,x and d ε,y Respectively represent the path X i and path X α The difference between the x-coordinate and the y-coordinate of the εth node;

[0101] A random vector is introduced to calculate the similarity difference between the coordinates, and the coordinates of each node in the i-th path are updated synchronously, which is expressed as:

[0102]

[0103] C = 2 rand(0,1);

[0104] in and They represent the x-coordinate and y-coordinate of the j-th node of path i at iteration number t+1, and They represent the x-coordinate and y-coordinate of the j-th node of path α with iteration number t, respectively. and They represent the x-coordinate and y-coordinate of the j-th node of path β with iteration number t, and They represent the x-coordinate and y-coordinate of the j-th node of the path δ with iteration number t, respectively. C represents a random vector, and the value of the random vector is between [0,2].

[0105] The fitness value of the path is recalculated in each iteration, and the position of the adjusted path is updated based on the paths with the best, second best and third best fitness values ​​of the current iteration as reference paths;

[0106] Based on the changes of the optimal, suboptimal and third-optimal fitness of the current iteration t and the previous iteration t-1, the change rates are calculated respectively, and the average value of the optimal, suboptimal and third-optimal fitness change rates is calculated as the comprehensive change rate, the comprehensive change rate of the previous s iterations is counted, and the average value of the comprehensive change rate of the previous s iterations is calculated as the adaptive threshold;

[0107] If the comprehensive change rate after each iteration is greater than or equal to the adaptive threshold, the iteration is stopped and the optimal fitness path of the current iteration is output as the optimal path.

[0108] By expanding multiple candidate paths based on the initial path and combining these candidate paths with the initial path into a candidate solution set, more possible path choices can be provided for subsequent optimization, effectively avoiding falling into the local optimal solution and improving the global search capability. By comprehensively considering the total length of the path and the number of obstacles passed through to calculate the fitness function, the path length and obstacle avoidance effect can be optimized at the same time, taking into account the safety and efficiency of the path. The introduction of the fitness function enables path optimization to effectively balance between length and the number of obstacles, select the path that best suits the actual scenario, and improve the robustness of path planning and the multi-objective optimization effect. By adopting a dynamic The position update strategy can continuously adjust the position of the candidate path in each iteration to gradually approach the optimal path. By introducing random vectors and adjusting the coefficient A based on the number of iterations, the path update can conduct a large-scale exploration in the early stage and gradually converge to the precise optimal path in the later stage, improving the global search capability and convergence speed of path planning. The introduction of the adaptive threshold enables the algorithm to adaptively adjust the number of iterations in scenes of different complexity, ensuring sufficient optimization effect in complex scenes. By calculating the similarity difference between the path node coordinates, the position update strategy is refined to each path node, improving the accuracy of path update.

[0109] This method uses A * The algorithm is extended to generate multiple candidate paths, and the initial path is combined with the candidate path to form a candidate solution set. Compared with the traditional gray wolf optimization algorithm, the gray wolf optimization can find a solution closer to the global optimal solution in multiple paths, which improves the overall performance of path planning. * The algorithm generates an initial path, and on this basis, the gray wolf optimization algorithm is used to generate multiple candidate paths, forming a variety of path choices and improving the flexibility of path planning. * The algorithm is good at global path planning, but it is easy to fall into the local optimal solution. The gray wolf optimization algorithm has good global search and local fine adjustment capabilities. The combined use of the two can be based on the preliminary path. Through the gray wolf optimization, the path can be searched globally and adjusted locally, avoiding the path from falling into the local optimal solution and improving the global optimality of path planning.* The initial path generated by the algorithm and the dynamic adjustment process of the candidate path during the gray wolf optimization process can visualize the path planning results, intuitively display the path selection and adjustment process of the AGV robot at different stages, and improve the interpretability of path planning. This joint strategy can significantly improve the efficiency and optimization effect of path planning, especially in complex environments. It can quickly find a high-quality initial solution, and further optimize it on this basis to improve the performance of the global path.

[0110] S3, construct the AGV robot state vector and set the AGV robot action, update and iterate the Q value based on the DQN model through the Bellman equation, and train the DQN model;

[0111] Preferably, constructing the AGV robot state vector and setting the AGV robot action, updating and iterating the Q value based on the DQN model through the Bellman equation, and training the DQN model, including:

[0112] Based on the gray wolf optimization algorithm, the optimal fitness path is used as the optimal path, and the state vector S is constructed through the state data of the AGV robot. t , set the AGV robot actions to include forward θ1, left turn θ2, right turn θ3, deceleration θ4 and stop θ5, and define the immediate reward r of the action w Including making the AGV robot approach the target point coordinates, avoid obstacles, collide with obstacles and reach the target point coordinates;

[0113] Based on the DQN model, the Q value of each possible action is calculated, and the action with the largest Q value is selected for execution, which is expressed as:

[0114]

[0115] In the formula, Q(s w ,θ w ) represents the Q value of executing action θ in state s at the current time w, r w represents the immediate reward of the action at time w, represents the discount factor for time w, Q(s w+1 ,θ' w+1 ) represents any action θ' in the state s at time w+1 w+1 The corresponding Q value is, represents the maximum discount factor, d w Indicates the distance between the AGV robot and the target point coordinates at time w, d max Indicates the distance from the starting point coordinates to the target point coordinates of the AGV robot;

[0116] Through the Bellman equation, the Q value is updated and iterated, expressed as:

[0117]

[0118] In the formula, φ represents the learning rate;

[0119] During the iterative update process, the current state vector, action, reward, and next state vector are recorded and stored in the experience replay pool;

[0120] Based on the stored data in the experience replay pool as training data, the DQN network is trained by the mean square error, which is expressed as:

[0121]

[0122] Where Loss represents the error value of the error function, N represents the total number of training data, and r i Indicates the instant reward for the action of item i, represents the discount factor at time i, Q(s i+1 ,θ) represents the Q value corresponding to each action θ of state s at time i+1, Q(s i ,θ i ) represents the execution action θ at the current time u in state s i Q value;

[0123] The model parameters are iteratively optimized through the Adam optimizer and the gradient descent method. When the calculated error value no longer decreases significantly during the continuous iteration process, the iteration is stopped and the model parameters are output to update the DQN model.

[0124] The DQN model calculates the Q value of each possible action and selects the action with the largest Q value for execution. It can dynamically adjust the motion path of the AGV robot to achieve autonomous obstacle avoidance, path optimization and other functions. Compared with traditional rule or model prediction methods, DQN can automatically learn the optimal strategy in a complex and changing environment, improving the intelligence of path planning and execution. The state vector is constructed through the state data of the AGV robot (such as current position, speed, acceleration, direction angle, etc.), which can more comprehensively describe the current operating status of the AGV robot. The DQN model based on the state vector can more accurately evaluate the value of different actions, thereby selecting the optimal motion strategy and avoiding path selection errors caused by incomplete state information. , by balancing the distribution of training data in the experience replay pool, the model can make full use of historical data for learning during training, avoid overfitting in specific scenarios, and improve the generalization ability of the model. By iteratively updating the Q value through the Bellman equation, the action strategy of the AGV robot can be continuously adjusted in a dynamic environment, so that the model can achieve real-time adaptive adjustment in a complex and changing environment. The Adam optimizer can adaptively adjust the learning rate, effectively avoiding the problem of gradient disappearance or gradient explosion, so that the model can also converge stably in a complex environment. The AGV robot control method of the DQN model can achieve intelligent path optimization and goal-oriented autonomous navigation through dynamic learning path planning and obstacle avoidance strategies. The AGV robot can balance the path selection between the target point and obstacles, effectively improving the accuracy of path planning and the flexibility of autonomous navigation;

[0125] By A * The algorithm performs global path search based on the comprehensive cost function and uses the gray wolf optimization through the global search mechanism. Finally, based on the DQN model combined with the real-time status data of the AGV robot, it can accurately avoid obstacles and adjust the path according to the real-time perceived information in the actual environment. * The algorithm and the Gray Wolf Optimization Algorithm provide a high-quality global path foundation for the DQN model. The DQN model further improves the accuracy and environmental adaptability of the local path, achieving a perfect combination of global optimality and local precision. Through the joint use of the Gray Wolf Optimization Algorithm and the DQN model, the AGV robot can achieve real-time response and autonomous learning in a complex dynamic environment. The Gray Wolf Optimization Algorithm dynamically adjusts the path globally, and the DQN model realizes intelligent obstacle avoidance and path fine-tuning locally, enabling the AGV robot to flexibly respond to various changes in the environment and improve the real-time and autonomous learning capabilities of path planning.

[0126] S4, conduct scenario testing on the DQN model and deploy it to the AGV robot, securely encrypt the collection, transmission and storage of AGV robot data, and back up the AGV robot data;

[0127] Preferably, the DQN model is scenario tested and deployed to the AGV robot, including:

[0128] The trained DQN model is deployed in a simulation environment for scenario testing. The sensor parameters are modified through human intervention to detect and identify the execution actions of the AGV robot. At the same time, the environmental parameters of the simulation environment, including temperature, humidity and light intensity, are adjusted to test the execution actions of the AGV robot in different environments. Safety strategies are set for the AGV robot, including braking and parking, and the DQN model is deployed to the AGV robot.

[0129] By manually intervening to modify sensor parameters (such as the resolution and refresh rate of lidar and cameras), we can simulate sensor data anomalies or equipment failures, test the DQN model's ability to respond and action selection strategy when receiving inaccurate or incomplete data, and adjust the environmental parameters in the simulated environment (such as temperature, humidity, light intensity, etc.) to test the behavior of the AGV robot under different external environmental conditions and evaluate its adaptability to harsh or complex environments. Dynamic environmental testing can help discover the model's weaknesses under different environmental parameters and make targeted improvements, thereby improving the model's adaptability and robustness to environmental changes and ensuring that the AGV robot can perform tasks stably and safely under various environmental conditions. By setting safety strategies such as braking and parking for the AGV robot, it can effectively prevent the AGV robot from being in danger in unexpected situations.

[0130] Furthermore, the collection, transmission and storage of AGV robot data are securely encrypted and transmitted, and AGV robot data is backed up, including:

[0131] Use AES encryption interface to collect data from AGV robots, and adopt timestamp mechanism to record the time information of data generation and transmission;

[0132] The TLS protocol is used to encrypt data transmission, and the transmitted data is symmetrically encrypted using AES-256, and backed up and stored in the cloud server and storage hard disk.

[0133] By collecting data through the AES encryption interface and encrypting the transmission process with the TLS protocol, we can prevent problems such as packet loss and tampering during data transmission, ensure data integrity, and thus improve the stability of the entire AGV robot system. Encrypted transmission ensures the reliability of data in complex network environments, avoids data corruption or tampering, and improves the stability of the system during long-term operation. By using AES encryption in the three links of data collection, transmission, and storage, we can achieve multi-level encryption protection, further enhancing the overall security of the data.

[0134] Example 2, reference Figure 2, which is the second embodiment of the present invention, and which is different from the previous embodiment, provides a system for an AGV robot operation control method, including:

[0135] The data acquisition module collects static and dynamic data of the environment where the AGV robot is located, and obtains the status data of the AGV robot;

[0136] The path generation module uses A * The algorithm constructs an initial path and determines the global path from the starting point to the target point;

[0137] The path optimization module uses the gray wolf optimization algorithm to generate multiple candidate paths based on the initial path, and selects the optimal path through the position update strategy;

[0138] The action strategy module uses the DQN model to train the action strategy of the AGV robot, updates the Q value through the Bellman equation, and updates the model parameters through the mean square error and Adam optimizer;

[0139] The scenario testing module deploys the trained DQN model to the AGV robot and performs scenario testing in a simulated environment;

[0140] The data security encryption module performs secure encryption processing on the data collection, transmission and storage of the AGV robot.

[0141] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0143] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0144] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An AGV robot operation control method, characterized in that: include, Collect environmental data based on sensors, obtain status data through AGV robots, and use A based on map data * The algorithm constructs the initial path; Use the Gray Wolf Optimization Algorithm to generate candidate paths, adjust the paths to update the positions and output the optimal paths; Construct the AGV robot state vector and set the AGV robot action, update and iterate the Q value based on the DQN model through the Bellman equation, and train the DQN model; The DQN model is tested in scenarios and deployed to the AGV robot. The collection, transmission and storage of AGV robot data are securely encrypted and the AGV robot data is backed up.

2. The AGV robot operation control method according to claim 1, characterized in that: The sensor-based environmental data collection and the AGV robot-based state data acquisition include: Obtain static and dynamic data of the environment where the AGV robot is located through lidar, camera and ultrasonic sensor; The static data includes the position data of fixed obstacles in the environment acquired by the laser radar sensor and constructing two-dimensional grid map data, and marking the coordinates of the fixed obstacles in the map data; Dynamic data includes real-time updates of the relative positions and velocities of dynamic objects in the environment detected by cameras and lidar; Based on the sensors inside the AGV robot, the robot obtains its own status data, including the current coordinate position, speed, acceleration and direction angle; Define the starting point coordinates and target point coordinates of the AGV robot and represent them in the grid map data.

3. The AGV robot operation control method according to claim 2, characterized in that: The map data is based on the use of A * The algorithm constructs the initial path, including, Based on the two-dimensional grid map data, the two-dimensional grid map data is marked as passable or impassable according to the coordinates of fixed obstacles and the coordinates of dynamic objects in the detection environment; Use A * The algorithm calculates the Euclidean distance h(x,y) based on the current position coordinates of the AGV robot and the target point coordinates; Based on the starting point coordinates of the AGV robot, define the moving distance from the starting point coordinates to the current node, and express the distance from the current position coordinates of the AGV robot to the starting point coordinates as g(x,y); Calculate the sum of h(x,y) and g(x,y) to represent A * The comprehensive cost of the algorithm f(x,y); Based on A * The algorithm creates an open list and a closed list. The open list stores the set of nodes to be expanded. Each node records the current coordinates, the current g(x,y) value, h(x,y) value and f(x,y) comprehensive cost value. The closed list records the coordinates of nodes that have been expanded. Select the node coordinate with the smallest comprehensive cost f(x,y) as the current position coordinate C, and move the node coordinate C from the open list to the closed list; If the current position coordinate C is not the target point coordinate, continue to expand the adjacent nodes, check the four adjacent nodes of the current position coordinate C, calculate the adjacent node g(x,y) value, h(x,y) value and f(x,y) comprehensive cost value, and move the expanded node coordinates to the closed list; If the current position coordinate C is the target point coordinate, then the node before the node extension is traced until it is traced to the starting point coordinate, and the node coordinates in the tracing process are marked as the initial path of the AGV robot.

4. The AGV robot operation control method according to claim 3, characterized in that: The method of using the gray wolf optimization algorithm to generate candidate paths, adjusting the paths to update the positions and outputting the optimal paths includes: Use the gray wolf optimization algorithm to set the number of wolves n, select the first n-1 node coordinates greater than the minimum value of the comprehensive cost f(x, y) according to the number of wolves n-1 as the current position coordinates D, and repeat the expansion of adjacent nodes to form a candidate path for the AGV robot, and combine the candidate path and the initial path of the AGV robot into a candidate solution set; Based on the total path length and the number of obstacles passed by the candidate solution path, the fitness of each candidate solution is calculated, which is expressed as: X={(x1,y1),(x2,y2),Λ,(x ε ,y ε )}; In the formula, f(X i ) represents the fitness value of the ith path in the candidate solution set, L(X i ) represents the total length of the i-th path, C(X i ) represents the number of obstacles passed by the i-th path, X represents the vector of the path, including the coordinates of all nodes of the path, x ε and ε Respectively represent the x-coordinate and y-coordinate of the ε-th node of path X; The fitness of different paths is calculated and sorted, and the top three paths are selected as reference paths, and the remaining paths are selected as adjustment paths for position update, which is expressed as: D(X α ,X i )={(d 1,x ,d 1,y ),(d 2,x ,d 2,y ),Λ,(d ε,x ,d ε,y )}; Where, X i (t+1) represents the i-th path with iteration number t+1, X α (t), X β (t) and X δ (t) represent the optimal path α, the second-best path β and the third-best path δ of iteration number t, A represents the adjustment coefficient based on the iteration number, D(X α ,X i ) represents the difference direction of the coordinates of each node between the optimal path α and the i-th path, T represents the maximum number of iterations, d ε,x and d ε,y Respectively represent the path X i and path X α The difference between the x-coordinate and y-coordinate of the εth node; The fitness value of the path is recalculated in each iteration, and the position of the adjusted path is updated based on the paths with the best, second best and third best fitness values ​​of the current iteration as reference paths; Based on the changes of the optimal, suboptimal and third-optimal fitness of the current iteration t and the previous iteration t-1, the change rates are calculated respectively, and the average value of the optimal, suboptimal and third-optimal fitness change rates is calculated as the comprehensive change rate, the comprehensive change rate of the previous s iterations is counted, and the average value of the comprehensive change rate of the previous s iterations is calculated as the adaptive threshold; If the comprehensive change rate after each iteration is greater than or equal to the adaptive threshold, the iteration is stopped and the optimal fitness path of the current iteration is output as the optimal path.

5. The AGV robot operation control method according to claim 4, characterized in that: The AGV robot state vector is constructed and the AGV robot action is set, the Q value is updated and iteratively calculated through the Bellman equation based on the DQN model, and the DQN model is trained, including: Based on the gray wolf optimization algorithm, the optimal fitness path is used as the optimal path, and the state vector S is constructed through the state data of the AGV robot. t , set the AGV robot actions to include forward θ1, left turn θ2, right turn θ3, deceleration θ4 and stop θ5, and define the immediate reward r of the action w Including making the AGV robot approach the target point coordinates, avoid obstacles, collide with obstacles and reach the target point coordinates; Based on the DQN model, the Q value of each possible action is calculated, and the action with the largest Q value is selected for execution, which is expressed as: In the formula, Q(s w ,θ w ) represents the Q value of executing action θ in state s at the current time w, r w represents the immediate reward of the action at time w, represents the discount factor for time w, Q(s w+1 ,θ' w+1 ) represents any action θ' in the state s at time w+1 w+1 The corresponding Q value is, represents the maximum discount factor, d w Indicates the distance between the AGV robot and the target point coordinates at time w, d max Indicates the distance from the starting point coordinates to the target point coordinates of the AGV robot; The Q value is calculated through the Bellman equation. During the iterative update process, the current state vector, action, reward, and next state vector are recorded and stored in the experience replay pool. Based on the stored data in the experience replay pool as training data, the DQN network is trained through the mean square error, and the model parameters are iteratively optimized through the Adam optimizer and the gradient descent method. When the calculated error value no longer decreases significantly during the continuous iteration process, the iteration is stopped and the model parameters are output to update the DQN model.

6. The AGV robot operation control method according to claim 5, characterized in that: The scenario testing of the DQN model and its deployment to the AGV robot includes: The trained DQN model is deployed in a simulation environment for scenario testing. The sensor parameters are modified through human intervention to detect and identify the execution actions of the AGV robot. At the same time, the environmental parameters of the simulation environment, including temperature, humidity and light intensity, are adjusted to test the execution actions of the AGV robot in different environments. Safety strategies are set for the AGV robot, including braking and parking, and the DQN model is deployed to the AGV robot.

7. The AGV robot operation control method according to claim 6, characterized in that: The collection, transmission and storage of AGV robot data are securely encrypted and transmitted, and the AGV robot data is backed up, including: Use AES encryption interface to collect data from AGV robots, and adopt timestamp mechanism to record the time information of data generation and transmission; The TLS protocol is used to encrypt data transmission, and the transmitted data is symmetrically encrypted using AES-256, and backed up and stored in the cloud server and storage hard disk.

8. A system based on the AGV robot operation control method according to any one of claims 1 to 7, characterized in that: include, The data acquisition module collects static and dynamic data of the environment where the AGV robot is located, and obtains the status data of the AGV robot; The path generation module uses A * The algorithm constructs an initial path and determines the global path from the starting point to the target point; The path optimization module uses the gray wolf optimization algorithm to generate multiple candidate paths based on the initial path, and selects the optimal path through the position update strategy; The action strategy module uses the DQN model to train the action strategy of the AGV robot, updates the Q value through the Bellman equation, and updates the model parameters through the mean square error and Adam optimizer; The scenario testing module deploys the trained DQN model to the AGV robot and performs scenario testing in a simulated environment; The data security encryption module performs secure encryption processing on the data collection, transmission and storage of the AGV robot.

9. A computer device comprising: A memory and a processor; the memory stores a computer program, wherein the processor implements the steps of the AGV robot operation control method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the AGV robot operation control method according to any one of claims 1 to 7 are implemented.

Images