AGV path planning method and device

Abstract

The invention discloses an AGV path planning method and device. The method comprises the following steps: acquiring environment information and state information of an AGV in real time through a plurality of sensors; the improved feature extraction model is used for carrying out feature extraction on the environment information and the state information to obtain feature information, and the improved feature extraction model comprises a mirror image expansion module; performing information fusion on the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; and an AGV track control instruction is generated according to the three-dimensional position information, the advancing speed and direction of the AGV are adjusted according to the AGV track control instruction, and AGV adaptive path planning is realized. According to the AGV path planning method, the feature extraction model comprising the mirror image expansion module is used for performing feature extraction on the environment information and the state information, and then information fusion is performed on the feature information through adaptive complementary filtering and Kalman filtering, so that the three-dimensional position information is obtained, and the AGV path planning efficiency and accuracy are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an AGV path planning method and device. Background Art

[0002] Automated Guided Vehicle (AGV) refers to a transport vehicle with an automatic guidance system that can run between the starting point and the end point according to a specified route or an automatically generated route. It has certain safety protections and integrates sound, light, electricity, and computer technologies. It integrates the advanced theories and application technologies in today's science and technology field. It is widely used in flexible manufacturing systems and automated factories. It has many advantages such as high transportation efficiency, energy saving, reliable operation, and flexible transportation, which greatly improves the degree of automation and production efficiency.

[0003] For AGV, choosing the right and effective path can improve transportation efficiency and reduce transportation costs. Existing AGV path planning technologies, such as navigation methods based on static maps, are difficult to work efficiently in dynamic and complex environments. Due to the lack of real-time environmental perception capabilities, these methods may make path planning inefficient, leading to collisions or failure to adjust the path in time.

[0004] Therefore, there is an urgent need for an AGV path planning method that can effectively improve the efficiency and accuracy of AGV path planning. Summary of the invention

[0005] The main purpose of the present invention is to provide an AGV path planning method and device, aiming to solve the technical problem in the prior art that AGV path planning cannot quickly adapt to environmental changes, resulting in low efficiency and accuracy of AGV path planning.

[0006] To achieve the above object, the present invention provides an AGV path planning method, which comprises the following steps: Acquire the AGV's environmental and status information in real time through multiple sensors; Using an improved feature extraction model to extract features from the environment information and the state information to obtain feature information, the improved feature extraction model comprising a mirror expansion module; The feature information is fused by adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; An AGV trajectory control instruction is generated according to the three-dimensional position information, and the travel speed and direction of the AGV are adjusted according to the AGV trajectory control instruction to realize AGV adaptive path planning.

[0007] Optionally, the step of generating an AGV trajectory control instruction according to the three-dimensional position information includes: Constructing a three-dimensional map of the environment according to the three-dimensional position information; Inputting the three-dimensional position information into a preset strategy optimization model to generate a plurality of action sequences to be executed and corresponding value evaluation values, wherein the preset strategy optimization model includes a strategy network and a value network, and the strategy network and the value network include a self-attention mechanism; Determine a target action sequence based on the three-dimensional map of the environment, the sequence of actions to be performed, and the value assessment value; According to the target action sequence, an AGV trajectory control instruction is generated.

[0008] Optionally, the step of inputting the three-dimensional position information into a preset strategy optimization model to generate a plurality of action sequences to be executed and corresponding value assessment values ​​includes: Performing feature enhancement on the three-dimensional position information through the self-attention mechanism to obtain feature-enhanced three-dimensional position information; Using the policy network, generating a plurality of action sequences to be executed based on the feature-enhanced three-dimensional position information; The value network is used to perform value evaluation on each of the to-be-executed action sequences to obtain a value evaluation value.

[0009] Optionally, the sensor includes a 2D laser radar, a binocular camera and an inertial measurement unit. Before the step of acquiring the environmental information and status information of the AGV in real time through the multiple sensors, the step further includes: Performing coordinate transformation calibration on the point cloud data of the 2D laser radar; Calibrate the binocular camera using Zhang's calibration method; The Dogleg algorithm and the Allan variance principle are used to calibrate the deterministic and random errors in the trust region of the inertial measurement unit.

[0010] Optionally, the step of acquiring the environmental information and status information of the AGV in real time through multiple sensors includes: The 2D laser radar is used to obtain the distance data and reflection intensity information of the three-dimensional space objects in the current environment in real time; Acquire stereoscopic visual information in the current environment through the binocular camera; Using the gyroscope and accelerometer of the inertial measurement unit to measure the angular velocity and acceleration of the AGV respectively, and using the angular velocity and acceleration of the AGV to infer the motion trajectory and posture change of the AGV; The stereoscopic vision information, the three-dimensional space object distance data and the reflection intensity information are used as the environmental information of the AGV, and the motion trajectory and posture change of the AGV are used as the state information of the AGV.

[0011] Optionally, after the step of calibrating the deterministic error and the random error using the Dogleg algorithm and the Allen variance principle in the trust domain of the inertial measurement unit, the method further comprises: Determining the coordinate systems of the binocular camera and the inertial measurement unit; Based on the coordinate systems of the binocular camera and the inertial measurement unit, a time synchronization process is performed using the Kalibr calibration tool to obtain the position and posture changes of the binocular camera and the inertial measurement unit after the time synchronization process; According to the change in the position and posture, external parameter information of the binocular camera and the inertial measurement unit is determined.

[0012] Optionally, after the step of calibrating the deterministic error and the random error using the Dogleg algorithm and the Allen variance principle in the trust domain of the inertial measurement unit, the method further comprises: Determine the coordinate system of the binocular camera and the 2D laser radar; Based on the coordinate systems of the binocular camera and the 2D laser radar, a nonlinear least squares method is constructed by using preset constraint equations; The nonlinear least squares method is used to determine the extrinsic parameter information of the 2D laser radar based on the extrinsic parameter information of the binocular camera.

[0013] Optionally, the step of fusing the feature information by adaptive complementary filtering and Kalman filtering to obtain the three-dimensional position information includes: Based on the three-dimensional space object distance data and reflection intensity information, a two-dimensional plane position is obtained by a position estimation algorithm; Using Kalman filtering to fuse the ultrasonic module data of the optical flow sensor and the data acquired by the inertial measurement unit to obtain initial three-dimensional position information; The two-dimensional plane position and the initial three-dimensional position information are fused through adaptive complementary filtering to obtain three-dimensional position information.

[0014] In addition, to achieve the above-mentioned purpose, the present invention also proposes an AGV path planning device, the device comprising: The information acquisition module is used to obtain the environmental information and status information of the AGV in real time through multiple sensors; A feature extraction module, used to extract features from the environment information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module; An information fusion module, used for fusing the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; The path planning module is used to generate an AGV trajectory control instruction according to the three-dimensional position information, adjust the travel speed and direction of the AGV according to the AGV trajectory control instruction, and realize real-time control of the AGV adaptive path planning.

[0015] The path planning module is also used to construct a three-dimensional map of the environment based on the three-dimensional position information; input the three-dimensional position information into a preset strategy optimization model to generate multiple action sequences to be executed and corresponding value assessment values, the preset strategy optimization model includes a strategy network and a value network, and the strategy network and the value network include a self-attention mechanism; determine the target action sequence based on the three-dimensional map of the environment, the action sequence to be executed and the value assessment value; generate AGV trajectory control instructions according to the target action sequence.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also proposes an AGV path planning device, which includes: a memory, a processor, and an AGV path planning program stored in the memory and executable on the processor, and the AGV path planning program is configured to implement the steps of the AGV path planning method described above.

[0017] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which an AGV path planning program is stored, and when the AGV path planning program is executed by a processor, the steps of the AGV path planning method described above are implemented.

[0018] The present invention discloses a method for obtaining environmental information and state information of an AGV in real time through multiple sensors; performing feature extraction on the environmental information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module; performing information fusion on the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; generating an AGV trajectory control instruction according to the three-dimensional position information, adjusting the travel speed and direction of the AGV according to the AGV trajectory control instruction, and realizing AGV adaptive path planning. Since the present invention uses a feature extraction model including a mirror expansion module to perform feature extraction on environmental information and state information, and then performs information fusion on the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information, and finally generates an AGV trajectory control instruction according to the three-dimensional position information, compared with the prior art, the present invention effectively improves the efficiency and accuracy of AGV path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the first embodiment of the AGV path planning method of the present invention; Figure 2This is a flow chart of a second embodiment of the AGV path planning method of the present invention; Figure 3 This is a flow chart of a third embodiment of the AGV path planning method of the present invention; Figure 4 It is a structural block diagram of the first embodiment of the AGV path planning device of the present invention; Figure 5 It is a structural diagram of an AGV path planning device in a hardware operating environment involved in an embodiment of the present invention.

[0020] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0022] The embodiment of the present invention provides an AGV path planning method, referring to Figure 1 , Figure 1 It is a flowchart of the first embodiment of the AGV path planning method of the present invention.

[0023] In this embodiment, the AGV path planning method includes the following steps S10-S40: Step S10: Acquire the environmental information and status information of the AGV in real time through multiple sensors.

[0024] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication and program running functions used in the AGV path planning scenario, such as a server, a tablet computer, a smart phone, a smart watch, etc., or an electronic device capable of realizing the above functions, an AGV path planning device, etc. The following takes the AGV path planning device as an example to illustrate this embodiment and the following embodiments.

[0025] It should be explained that the above-mentioned sensors include 2D lidar, binocular camera and inertial measurement unit.

[0026] It should be noted that 2D LiDAR can obtain high-precision distance data and reflection intensity information of three-dimensional space objects, which can be used to build a three-dimensional map of the surrounding environment. Binocular cameras can observe the same object from two different viewpoints at the same time, obtain the pixel difference (i.e., parallax) of the corresponding points of the same image imaged by the left and right cameras, and use the principle of triangulation to calculate the depth information of the object, thereby obtaining stereoscopic visual information of the environment. The inertial measurement unit (IMU) can use gyroscopes and accelerometers to measure the angular velocity and acceleration of the AGV car, and these data are used to infer the motion trajectory and posture changes of the AGV car.

[0027] It should be explained that the above AGV environment information may include stereoscopic vision information in the current environment, three-dimensional space object distance data and reflection intensity information in the current environment. The above AGV state information may include the movement trajectory and posture change of the AGV.

[0028] In a specific implementation, the 2D laser radar can be used to obtain the three-dimensional space object distance data and reflection intensity information in the current environment in real time; the stereoscopic vision information in the current environment can be obtained through the binocular camera; the gyroscope and accelerometer of the inertial measurement unit are used to measure the angular velocity and acceleration of the AGV respectively, and the angular velocity and acceleration of the AGV are used to infer the motion trajectory and posture changes of the AGV; the stereoscopic vision information, the three-dimensional space object distance data and reflection intensity information are used as the environmental information of the AGV, and the motion trajectory and posture changes of the AGV are used as the status information of the AGV.

[0029] Step S20: extracting features from the environment information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module.

[0030] It should be noted that the improved feature extraction model is a model obtained by improving the ORB (Oriented FAST and Rotated BRIEF) feature extraction model by adding a mirror expansion module. Since the ORB feature extraction model is a fast feature extraction algorithm, this embodiment improves the feature extraction model by adding a step of mirror expansion of undetectable pixels.

[0031] Furthermore, this embodiment proposes a mirror expansion method to solve the problem that the existing ORB feature extraction model is difficult to extract feature points at the edge of the image. Specifically, when performing feature point detection, the three pixel points at the edge of the image are mirror expanded, so that feature points that were originally undetectable in the edge area of ​​the image can be detected. Among them, the improvement of the feature point detection process: the improved feature point detection process includes the following steps: mirror expanding the three pixel points at the edge of the image; selecting grayscale values ​​in the image Pixels ; Set a threshold ; In pixels As the center, select 16 pixels on a circle with a radius of 3 pixels; if the selected 16 pixels have Continuous gray values ​​greater than or less than , then the point is considered to be a feature point.

[0032] It is understandable that the above grayscale values and threshold It can be a user-defined setting, and this embodiment does not limit this.

[0033] In the specific implementation, the improved feature extraction model and LK optical flow algorithm can be used to calculate the speed of the AGV.

[0034] Step S30: fusing the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information.

[0035] It should be understood that adaptive complementary filtering combines the measurement data of multiple sensors and automatically adjusts the filtering parameters according to the characteristics of these data and environmental changes to obtain the best filtering effect. This filtering method utilizes the complementarity of different sensors in measurement and can significantly improve measurement accuracy and stability by fusing their outputs.

[0036] Kalman filtering is an algorithm that uses the linear system state equation to optimally estimate the system state through the system input and output observation data. Since the observation data includes the influence of noise and interference in the system, the optimal estimate can also be regarded as a filtering process. Data filtering is a data processing technology that removes noise and restores real data. Kalman filtering can estimate the state of a dynamic system from a series of data with measurement noise when the measurement variance is known.

[0037] In a specific implementation, the two-dimensional plane position can be obtained through a position estimation algorithm based on the three-dimensional space object distance data and reflection intensity information; the ultrasonic module data of the optical flow sensor and the data obtained by the inertial measurement unit are fused using Kalman filtering to obtain initial three-dimensional position information; the two-dimensional plane position and the initial three-dimensional position information are fused through adaptive complementary filtering to obtain three-dimensional position information.

[0038] Step S40: Generate an AGV trajectory control instruction according to the three-dimensional position information, adjust the travel speed and direction of the AGV according to the AGV trajectory control instruction, and implement AGV adaptive path planning.

[0039] In the specific implementation, the three-dimensional position information can be used to build a three-dimensional map of the AGV's surrounding environment, identify obstacles, paths, and target locations in the environment, and then use the strategy optimization algorithm to adjust the path in real time to ensure the optimality and safety of the path, thereby realizing AGV adaptive path planning.

[0040] This embodiment discloses the method of obtaining the environmental information and state information of the AGV in real time through multiple sensors; extracting features from the environmental information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module; fusing the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; generating AGV trajectory control instructions according to the three-dimensional position information, adjusting the travel speed and direction of the AGV according to the AGV trajectory control instructions, and realizing AGV adaptive path planning. Since this embodiment uses a feature extraction model including a mirror expansion module to extract features from the environmental information and state information, and then fuses the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information, and finally generates AGV trajectory control instructions according to the three-dimensional position information, compared with the prior art, this embodiment effectively improves the efficiency and accuracy of AGV path planning.

[0041] refer to Figure 2 , Figure 2 Schematic diagram of the flow chart of the second embodiment of the AGV path planning method of the present invention.

[0042] Based on the first embodiment, in this embodiment, step S40 includes steps S401 to S404: Step S401: constructing a three-dimensional map of the environment according to the three-dimensional position information.

[0043] Step S402: Input the three-dimensional position information into a preset strategy optimization model to generate multiple action sequences to be executed and corresponding value assessment values. The preset strategy optimization model includes a strategy network and a value network, and the strategy network and the value network include a self-attention mechanism.

[0044] Step S403: determining a target action sequence based on the three-dimensional map of the environment, the action sequence to be executed and the value assessment value.

[0045] Step S404: Generate AGV trajectory control instructions according to the target action sequence.

[0046] It should be noted that the role of the self-attention mechanism is to improve the ability of the policy network and the value network to process input features, and to enhance feature representation by focusing on the more important parts of the input features, thereby improving decision-making ability and the accuracy of value assessment. Through the attention mechanism, the network can better understand environmental information and guide the AGV to make better decisions.

[0047] The core of the self-attention mechanism is to calculate the attention weights, which represent the degree of attention each element in the sequence pays to other elements. Specifically, the self-attention mechanism includes the following four steps: 1. Calculate the query, key, and value: For each element in the sequence, a query vector, a key vector, and a value vector are calculated; these vectors are usually obtained by multiplying the element representation (such as word embedding) by a learnable weight matrix. 2. Calculate the attention scores: For each element in the sequence, use its query vector to dot product with the key vectors of all elements in the sequence to obtain the attention score; these scores represent the similarity between the query vector and the key vector. 3. Apply the Softmax function: The attention scores are normalized by the Softmax function to obtain the attention weights; the Softmax function ensures that the sum of the attention weights of all elements is 1, thereby representing a probability distribution. 4. Weighted Sum: Use the attention weights to weight the value vectors of all elements in the sequence to obtain the final output representation; this output representation is a weighted average that integrates the information of all elements in the sequence.

[0048] It should be understood that in the policy network, the use of the self-attention mechanism to process the input features and generate enhanced feature representations can effectively improve the decision-making ability of the strategy. In the value network, the use of the self-attention mechanism to process the input features and generate enhanced feature representations can effectively improve the accuracy of value assessment.

[0049] It should be explained that the main task of the policy network is to learn a policy, that is, the probability distribution of selecting the best action under a given environmental state. It determines how the agent should move in a specific environment. In path planning, the policy network generates actions based on the current environmental state and the feature representation enhanced by the self-attention mechanism. These actions may include going straight, turning, accelerating or decelerating, etc., to achieve optimal path tracking and obstacle avoidance.

[0050] The value network is used to evaluate the expected return of the current strategy, that is, to predict the long-term benefits that may be obtained after adopting a certain strategy. This helps to determine whether the current strategy is good or bad and guides the update direction of the policy network. In path planning, the value network evaluates the potential value of the current path or action sequence, helping the agent understand which paths or actions are more likely to lead to successful arrival at the destination while avoiding potential obstacles and risks.

[0051] In the above preset policy optimization model, the policy network and the value network usually work together. The policy network is responsible for generating the probability of taking various actions in a given state, while the value network evaluates the expected returns of these states or action sequences. The two share environmental state information and update each other through joint training and feedback loops. The policy network uses the evaluation of the value network to optimize action selection, while the value network provides an evaluation of policy performance, helping the agent find a balance between exploring new actions and using known information, thereby achieving more efficient learning and more accurate decision-making.

[0052] In a specific implementation, the step of inputting the three-dimensional position information into a preset strategy optimization model to generate multiple action sequences to be executed and corresponding value assessment values ​​may include: performing feature enhancement on the three-dimensional position information through the self-attention mechanism to obtain the three-dimensional position information after feature enhancement; using the strategy network to generate multiple action sequences to be executed based on the three-dimensional position information after feature enhancement; performing value assessment on each of the action sequences to be executed through the value network to obtain a value assessment value.

[0053] This embodiment discloses obtaining the environmental information and state information of the AGV in real time through multiple sensors; extracting features from the environmental information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module; fusing the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; constructing a three-dimensional map of the environment according to the three-dimensional position information; inputting the three-dimensional position information into a preset strategy optimization model to generate multiple action sequences to be executed and corresponding value evaluation values, wherein the preset strategy optimization model includes a strategy network and a value network, wherein the strategy network and the value network include a self-attention mechanism; determining a target action sequence based on the three-dimensional map of the environment, the action sequence to be executed and the value evaluation value; and generating an AGV trajectory control instruction based on the target action sequence. Compared with the prior art, this embodiment effectively improves the safety and accuracy of path planning by inputting the three-dimensional position information into a preset strategy optimization model to generate multiple action sequences to be executed and corresponding value evaluation values, and then determining the target action sequence based on the three-dimensional map of the environment, the action sequence to be executed and the value evaluation value.

[0054] refer to Figure 3 , Figure 3 Schematic diagram of the flow chart of the third embodiment of the AGV path planning method of the present invention.

[0055] Based on the above embodiments, in this embodiment, before step S10, steps S01 to S03 are included: Step S01: performing coordinate transformation calibration on the point cloud data of the 2D laser radar.

[0056] Step S02: calibrating the binocular camera by using Zhang's calibration method.

[0057] Step S03: Calibrate the deterministic error and random error using the Dogleg algorithm and the Allan variance principle in the trust region of the inertial measurement unit.

[0058] It should be noted that after step S03, it also includes: determining the coordinate system of the binocular camera and the inertial measurement unit; based on the coordinate system of the binocular camera and the inertial measurement unit, using the Kalibr calibration tool to perform time synchronization processing to obtain the posture change of the binocular camera and the inertial measurement unit after the time synchronization processing; according to the posture change, determining the external parameter information of the binocular camera and the inertial measurement unit.

[0059] Furthermore, after step S03, it also includes: determining the coordinate systems of the binocular camera and the 2D laser radar; constructing a nonlinear least squares method based on the coordinate systems of the binocular camera and the 2D laser radar through a preset constraint equation; and using the nonlinear least squares method to determine the external parameter information of the 2D laser radar based on the external parameter information of the binocular camera.

[0060] It should be understood that this embodiment effectively improves the accuracy of subsequent sensor measurement data by performing coordinate transformation calibration of point cloud data of 2D lidar, calibrating the binocular camera, calibrating the deterministic error and random error of the trust domain of the inertial measurement unit, and determining the external parameter information of the binocular camera, inertial measurement unit and 2D lidar, thereby ensuring the reliability of AGV path planning.

[0061] This embodiment discloses the calibration of coordinate transformation of point cloud data of the 2D laser radar; the calibration of the binocular camera using Zhang's calibration method; the calibration of deterministic error and random error using Dogleg algorithm and Allen variance principle in the trust domain of the inertial measurement unit; the environmental information and state information of the AGV are acquired in real time through multiple sensors; the environmental information and the state information are extracted by an improved feature extraction model to obtain feature information, and the improved feature extraction model includes a mirror expansion module; the feature information is fused by adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; an AGV trajectory control instruction is generated according to the three-dimensional position information, and the travel speed and direction of the AGV are adjusted according to the AGV trajectory control instruction to realize AGV adaptive path planning. Compared with the prior art, this embodiment effectively improves the accuracy of sensor measurement data, thereby ensuring the reliability of AGV path planning.

[0062] In addition, an embodiment of the present invention further proposes a storage medium, on which an AGV path planning program is stored. When the AGV path planning program is executed by a processor, the steps of the AGV path planning method described above are implemented.

[0063] Reference Figure 4 , Figure 4 This is a structural block diagram of the first embodiment of the AGV path planning device of the present invention.

[0064] like Figure 4 As shown, the AGV path planning device proposed in the embodiment of the present invention includes: an information acquisition module 401, a feature extraction module 402, an information fusion module 403 and a path planning module 404.

[0065] The information acquisition module 401 is used to acquire the environment information and status information of the AGV in real time through multiple sensors.

[0066] The feature extraction module 402 is used to extract features from the environment information and the state information using an improved feature extraction model to obtain feature information. The improved feature extraction model includes a mirror expansion module.

[0067] The information fusion module 403 is used to fuse the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information.

[0068] The path planning module 404 is used to generate an AGV trajectory control instruction according to the three-dimensional position information, adjust the travel speed and direction of the AGV according to the AGV trajectory control instruction, and realize real-time control of AGV adaptive path planning.

[0069] The sensors include 2D lidar, binocular camera and inertial measurement unit.

[0070] The information acquisition module 401 is also used to acquire the three-dimensional space object distance data and reflection intensity information in the current environment in real time through the 2D laser radar; acquire the stereoscopic vision information in the current environment through the binocular camera; use the gyroscope and accelerometer of the inertial measurement unit to respectively measure the angular velocity and acceleration of the AGV, and use the angular velocity and acceleration of the AGV to infer the motion trajectory and posture changes of the AGV; use the stereoscopic vision information, the three-dimensional space object distance data and reflection intensity information as the environmental information of the AGV, and use the motion trajectory and posture changes of the AGV as the status information of the AGV.

[0071] The information fusion module 403 is also used to obtain the two-dimensional plane position through a position estimation algorithm based on the three-dimensional space object distance data and reflection intensity information; use Kalman filtering to fuse the ultrasonic module data of the optical flow sensor and the data obtained by the inertial measurement unit to obtain initial three-dimensional position information; and use adaptive complementary filtering to fuse the two-dimensional plane position and the initial three-dimensional position information to obtain three-dimensional position information.

[0072] The embodiment of the present device discloses obtaining the environmental information and state information of the AGV in real time through multiple sensors; extracting features from the environmental information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module; fusing the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; generating AGV trajectory control instructions according to the three-dimensional position information, adjusting the travel speed and direction of the AGV according to the AGV trajectory control instructions, and realizing AGV adaptive path planning. Since the embodiment of the present device uses a feature extraction model including a mirror expansion module to extract features from the environmental information and state information, and then fuses the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information, and finally generates AGV trajectory control instructions according to the three-dimensional position information, compared with the prior art, the embodiment of the present device effectively improves the efficiency and accuracy of AGV path planning.

[0073] Based on the above-mentioned first embodiment of the AGV path planning device of the present invention, a second embodiment of the AGV path planning device of the present invention is proposed.

[0074] In this embodiment, the path planning module 404 is also used to construct a three-dimensional map of the environment based on the three-dimensional position information; input the three-dimensional position information into a preset strategy optimization model to generate multiple action sequences to be executed and corresponding value assessment values, the preset strategy optimization model includes a strategy network and a value network, and the strategy network and the value network include a self-attention mechanism; determine the target action sequence based on the three-dimensional map of the environment, the action sequence to be executed and the value assessment value; generate AGV trajectory control instructions according to the target action sequence.

[0075] The path planning module 404 is also used to perform feature enhancement on the three-dimensional position information through the self-attention mechanism to obtain the three-dimensional position information after feature enhancement; use the strategy network to generate multiple action sequences to be executed based on the three-dimensional position information after feature enhancement; and perform value evaluation on each of the action sequences to be executed through the value network to obtain a value evaluation value.

[0076] Other embodiments or specific implementations of the AGV path planning device of the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0077] The present application provides an AGV path planning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the AGV path planning method in the above-mentioned embodiment one.

[0078] Reference below Figure 5 , which shows a schematic diagram of the structure of an AGV path planning device suitable for implementing the embodiment of the present application. The AGV path planning device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The AGV path planning device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0079] like Figure 5 As shown, the AGV path planning device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the AGV path planning device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the AGV path planning device to communicate wirelessly or wired with other devices to exchange data. Although the AGV path planning device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0080] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0081] The AGV path planning device provided by the present application adopts the AGV path planning method in the above embodiment, which can solve the technical problem that the AGV path planning in the prior art cannot quickly adapt to environmental changes, resulting in low efficiency and accuracy of AGV path planning. Compared with the prior art, the beneficial effects of the AGV path planning device provided by the present application are the same as the beneficial effects of the AGV path planning method provided by the above embodiment, and the other technical features in the AGV path planning device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0082] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0083] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0084] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0085] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0087] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An AGV path planning method, characterized in that: The method comprises: Acquire the AGV's environmental and status information in real time through multiple sensors; Using an improved feature extraction model to extract features from the environment information and the state information to obtain feature information, the improved feature extraction model comprising a mirror expansion module; The feature information is fused by adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; An AGV trajectory control instruction is generated according to the three-dimensional position information, and the travel speed and direction of the AGV are adjusted according to the AGV trajectory control instruction to realize AGV adaptive path planning.

2. The AGV path planning method according to claim 1, characterized in that: The step of generating an AGV trajectory control instruction according to the three-dimensional position information comprises: Constructing a three-dimensional map of the environment according to the three-dimensional position information; Inputting the three-dimensional position information into a preset strategy optimization model to generate a plurality of action sequences to be executed and corresponding value evaluation values, wherein the preset strategy optimization model includes a strategy network and a value network, and the strategy network and the value network include a self-attention mechanism; Determine a target action sequence based on the three-dimensional map of the environment, the sequence of actions to be performed, and the value assessment value; According to the target action sequence, an AGV trajectory control instruction is generated.

3. The AGV path planning method according to claim 2, characterized in that: The step of inputting the three-dimensional position information into a preset strategy optimization model to generate a plurality of action sequences to be executed and corresponding value assessment values ​​includes: Performing feature enhancement on the three-dimensional position information through the self-attention mechanism to obtain feature-enhanced three-dimensional position information; Using the policy network, generating a plurality of action sequences to be executed based on the feature-enhanced three-dimensional position information; The value network is used to perform value evaluation on each of the to-be-executed action sequences to obtain a value evaluation value.

4. The AGV path planning method according to claim 1, characterized in that: The sensors include a 2D laser radar, a binocular camera and an inertial measurement unit. Before the step of obtaining the environmental information and status information of the AGV in real time through multiple sensors, the method further includes: Performing coordinate transformation calibration on the point cloud data of the 2D laser radar; Calibrate the binocular camera using Zhang's calibration method; The Dogleg algorithm and the Allan variance principle are used to calibrate the deterministic and random errors in the trust region of the inertial measurement unit.

5. The AGV path planning method according to claim 4, characterized in that: The step of obtaining the environmental information and status information of the AGV in real time through multiple sensors includes: The 2D laser radar is used to obtain the distance data and reflection intensity information of the three-dimensional space objects in the current environment in real time; Acquire stereoscopic visual information in the current environment through the binocular camera; Using the gyroscope and accelerometer of the inertial measurement unit to measure the angular velocity and acceleration of the AGV respectively, and using the angular velocity and acceleration of the AGV to infer the motion trajectory and posture change of the AGV; The stereoscopic vision information, the three-dimensional space object distance data and the reflection intensity information are used as the environmental information of the AGV, and the motion trajectory and posture change of the AGV are used as the state information of the AGV.

6. The AGV path planning method according to claim 4, characterized in that: After the step of using the Dogleg algorithm and the Allen variance principle to calibrate the deterministic error and the random error in the trust domain of the inertial measurement unit, the method further includes: Determining the coordinate systems of the binocular camera and the inertial measurement unit; Based on the coordinate systems of the binocular camera and the inertial measurement unit, a time synchronization process is performed using the Kalibr calibration tool to obtain the position and posture changes of the binocular camera and the inertial measurement unit after the time synchronization process; According to the change in the position and posture, external parameter information of the binocular camera and the inertial measurement unit is determined.

7. The AGV path planning method according to claim 6, characterized in that: After the step of using the Dogleg algorithm and the Allen variance principle to calibrate the deterministic error and the random error in the trust domain of the inertial measurement unit, the method further includes: Determine the coordinate system of the binocular camera and the 2D laser radar; Based on the coordinate systems of the binocular camera and the 2D laser radar, a nonlinear least squares method is constructed by using preset constraint equations; The nonlinear least squares method is used to determine the extrinsic parameter information of the 2D laser radar based on the extrinsic parameter information of the binocular camera.

8. The AGV path planning method according to claim 5, characterized in that: The step of fusing the feature information by adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information includes: Based on the three-dimensional space object distance data and reflection intensity information, a two-dimensional plane position is obtained by a position estimation algorithm; Using Kalman filtering to fuse the ultrasonic module data of the optical flow sensor and the data acquired by the inertial measurement unit to obtain initial three-dimensional position information; The two-dimensional plane position and the initial three-dimensional position information are fused through adaptive complementary filtering to obtain three-dimensional position information.

9. An AGV path planning device, characterized in that: The device comprises: The information acquisition module is used to obtain the environmental information and status information of the AGV in real time through multiple sensors; A feature extraction module, used to extract features from the environment information and the state information using an improved feature extraction model to obtain feature information, wherein the improved feature extraction model includes a mirror expansion module; An information fusion module, used for fusing the feature information through adaptive complementary filtering and Kalman filtering to obtain three-dimensional position information; The path planning module is used to generate an AGV trajectory control instruction according to the three-dimensional position information, adjust the travel speed and direction of the AGV according to the AGV trajectory control instruction, and realize real-time control of the AGV adaptive path planning.

10. The AGV path planning device according to claim 9, characterized in that: The path planning module is further used to construct a three-dimensional map of the environment according to the three-dimensional position information; input the three-dimensional position information into a preset strategy optimization model to generate a plurality of action sequences to be executed and corresponding value evaluation values, wherein the preset strategy optimization model includes a strategy network and a value network, and the strategy network and the value network include a self-attention mechanism; A target action sequence is determined based on the three-dimensional map of the environment, the sequence of actions to be executed and the value assessment value; and an AGV trajectory control instruction is generated according to the target action sequence.

Images