A method, system and device for positioning an automatic guided vehicle based on deep reinforcement learning

Abstract

The present invention relates to an automatic guided vehicle positioning method, system and device based on deep reinforcement learning, and the method comprises the following steps: collecting historical motion data of the automatic guided vehicle; using the historical motion data to train a deep reinforcement learning model to obtain a movement deviation prediction model; collecting real-time motion data of the automatic guided vehicle; inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the vehicle head orientation angle of the automatic guided vehicle; calculating the position of the automatic guided vehicle according to the real-time motion data, the position deviation and the vehicle head orientation angle to realize the positioning of the automatic guided vehicle; the present invention can realize the application of the guide vehicle on a two-dimensional code path with large spacing and straight line paving through an algorithm for automatic deviation correction, and the position of the guide vehicle can also be accurately calculated, thereby improving the positioning accuracy of the guide vehicle.

Description

Technical Field

[0001] The present invention relates to the field of navigation technology, and in particular to a method, system and device for positioning an automatic guided vehicle based on deep reinforcement learning. Background Art

[0002] Automatic guided vehicles usually use inertial measurement units to measure the acceleration of the guided vehicles. Since the inertial measurement units have large drift errors after long-term operation, the existing two-dimensional code navigation, inertial navigation, and the fusion navigation that combines the two are highly dependent on the precise position of the two-dimensional code for navigation. Therefore, the spacing between the two-dimensional codes needs to be set smaller to avoid the situation where the automatic guided vehicle cannot scan the two-dimensional code during navigation. When the automatic guided vehicle travels on a two-dimensional code path with large spacing and straight lines, the navigation error of the automatic guided vehicle is large due to the large gap between two adjacent two-dimensional codes. Summary of the invention

[0003] In order to solve the technical problem in the prior art that when the automatic guided vehicle travels on a QR code path paved with large spacing and straight lines, the automatic guided vehicle navigation error is large, the present invention provides an automatic guided vehicle positioning method, system and device based on deep reinforcement learning.

[0004] The technical solution of the present invention to solve the above technical problems is as follows:

[0005] A method for positioning an automatic guided vehicle based on deep reinforcement learning comprises the following steps:

[0006] Collect historical movement data of automated guided vehicles;

[0007] Using the historical motion data to train a deep reinforcement learning model to obtain a movement deviation prediction model;

[0008] Collecting real-time motion data of the automatic guided vehicle;

[0009] Inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data;

[0010] The position of the automatic guided vehicle is calculated according to the real-time motion data and the position deviation prediction data to achieve positioning of the automatic guided vehicle.

[0011] The beneficial effects of the present invention are: by using a deep reinforcement learning model to predict the position deviation data of the automatic guided vehicle, and by calculating the actual position of the guided vehicle through the measured data and the position deviation data during the movement of the guided vehicle, that is, by compensating the real-time measurement data through the position deviation data, the accuracy of the position calculation of the guided vehicle can be improved; even if the guided vehicle is traveling on a path with a large spacing between two adjacent two-dimensional code roads, the position of the guided vehicle can be accurately calculated, thereby improving the positioning accuracy of the guided vehicle. Therefore, the present invention can realize the application of the automatic guided vehicle on the two-dimensional code path with large spacing and straight lines through the algorithm automatic deviation correction.

[0012] Based on the above technical solution, the present invention can also be improved as follows.

[0013] Further, the historical motion data includes historical acceleration data, historical moving speed data and historical vehicle head angle data of the automatic guided vehicle;

[0014] Collect historical movement data of the AGV, including:

[0015] Using an inertial measurement device to measure the acceleration of the automatic guided vehicle during driving, to obtain the historical acceleration data;

[0016] Using an odometer to measure the moving speed of the automatic guided vehicle during driving, to obtain the historical moving speed data;

[0017] The historical vehicle head angle data is obtained by scanning the QR code on the driving path using a camera installed on the automatic guided vehicle; wherein, there are multiple QR codes, and the multiple QR codes are distributed in a matrix on the driving path.

[0018] Furthermore, the deep reinforcement learning model is specifically a proximal strategy optimization model;

[0019] The historical motion data is used to train a deep reinforcement learning model to obtain a motion deviation prediction model, which specifically includes the following steps:

[0020] Constructing the deep reinforcement learning model based on the proximal strategy optimization algorithm to obtain an initial prediction model;

[0021] Using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to train the initial prediction model to obtain an initial strategy network model;

[0022] Using an inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving to obtain real-time acceleration data;

[0023] Using an odometer to measure the real-time moving speed of the automatic guided vehicle during driving to obtain real-time moving speed data;

[0024] Using a camera installed on the automatic guided vehicle to scan the QR code on the driving path in real time, to obtain real-time vehicle head angle data and real-time position deviation data;

[0025] Using the real-time acceleration data and the real-time moving speed data to input the initial strategy network model to predict initial position deviation prediction data and initial vehicle head angle prediction data of the automatic guided vehicle;

[0026] Comparing the difference between the real-time position deviation data and the initial position deviation prediction data to obtain a position prediction difference result;

[0027] Comparing the difference between the initial vehicle head angle prediction data and the real-time vehicle head angle data to obtain an angle prediction difference result;

[0028] The initial strategy network model is optimized using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model.

[0029] Further, the initial prediction model is trained using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to obtain an initial strategy network model, which specifically includes the following steps:

[0030] Splitting the historical acceleration data into acceleration training data and acceleration verification data;

[0031] Splitting the historical moving speed data into moving speed training data and moving speed verification data;

[0032] Using the acceleration training data, the moving speed training data and the historical vehicle head angle data to train the initial prediction model, to obtain the initial strategy network model;

[0033] The initial strategy network model is optimized by using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model, which specifically includes the following steps:

[0034] Calculate model optimization parameters according to the position prediction difference result and the angle prediction difference result;

[0035] The initial strategy network model is optimized using the model optimization parameters to obtain the movement deviation prediction model.

[0036] Further, the real-time motion data includes real-time acceleration data and the real-time moving speed data;

[0037] Collecting the real-time motion data of the automatic guided vehicle specifically includes the following steps:

[0038] Using the inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving, to obtain the real-time acceleration data;

[0039] The real-time moving speed of the automatic guided vehicle during driving is measured by using an odometer to obtain the real-time moving speed data.

[0040] Further, the real-time motion data is input into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data, specifically:

[0041] The real-time acceleration data and the real-time moving speed data are input into the moving deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle, and obtain the inertial measurement speed deviation value, the mileage measurement speed deviation value and the angle deviation value; wherein the position deviation prediction data includes the inertial measurement speed deviation value, the mileage measurement speed deviation value and the angle deviation value.

[0042] Further, the position of the automatic guided vehicle is calculated according to the real-time motion data and the position deviation prediction data to achieve positioning of the automatic guided vehicle, which specifically includes the following steps:

[0043] Using a Kalman filter algorithm to calculate the displacement value of the automatic guided vehicle according to the real-time motion data, the inertial measurement speed deviation value, and the mileage measurement speed deviation value;

[0044] Calculating the head deviation angle of the automatic guided vehicle according to the angle deviation value;

[0045] The position of the automatic guided vehicle is calculated according to the displacement value and the vehicle head offset angle to achieve positioning of the automatic guided vehicle.

[0046] Further, a Kalman filter algorithm is used to calculate the displacement value of the automatic guided vehicle according to the real-time motion data, the inertial measurement speed deviation value and the mileage measurement speed deviation value, which specifically includes the following steps:

[0047] Calculating a Kalman gain using the inertial measurement speed deviation value and the mileage measurement speed deviation value;

[0048] A Kalman filter algorithm is used to calculate the displacement value of the automatic guided vehicle according to the real-time motion data and the Kalman gain.

[0049] In order to solve the above technical problems, the present invention also provides an automatic guided vehicle positioning system based on deep reinforcement learning, and its specific technical contents are as follows:

[0050] An automatic guided vehicle positioning system based on deep reinforcement learning, comprising:

[0051] A data acquisition module, used to collect historical movement data of the automated guided vehicle;

[0052] A model building module, used to train a deep reinforcement learning model using the historical motion data to obtain a movement deviation prediction model;

[0053] The data acquisition module is also used to collect real-time motion data of the automatic guided vehicle;

[0054] A prediction processing module, used for inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data;

[0055] A position calculation module is used to calculate the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data, so as to realize the positioning of the automatic guided vehicle.

[0056] In order to solve the above technical problems, the present invention also provides an automatic guided vehicle positioning system based on deep reinforcement learning, and its specific technical contents are as follows:

[0057] An automatic guided vehicle positioning device includes a memory and one or more processors, wherein the memory stores executable code, and when one or more of the processors execute the executable code, the steps of the automatic guided vehicle positioning method based on deep reinforcement learning are implemented as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flowchart of an automatic guided vehicle positioning method based on deep reinforcement learning in an embodiment of the present invention;

[0059] Figure 2 It is a flowchart of a proximal strategy optimization algorithm in an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of the distribution of two-dimensional codes on the driving path of an automatic guided vehicle when collecting historical motion data in an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the distribution of QR codes on the driving path of an automatic guided vehicle in actual application;

[0062] Figure 5 A flowchart of using a Kalman filter algorithm to calculate the position of an automatic guided vehicle in an embodiment of the present invention;

[0063] Figure 6 This is a structural block diagram of an automatic guided vehicle positioning system based on deep reinforcement learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0065] Example 1

[0066] like Figure 1 As shown, a method for positioning an automatic guided vehicle based on deep reinforcement learning is provided, comprising the following steps:

[0067] S1. Collecting historical motion data of an automated guided vehicle; the historical motion data includes historical acceleration data, historical moving speed data, and historical vehicle head angle data of the automated guided vehicle;

[0068] Collect historical movement data of the AGV, including:

[0069] S101, using an inertial measurement device to measure the acceleration of the automatic guided vehicle during driving, to obtain the historical acceleration data;

[0070] S102, using an odometer to measure the moving speed of the automatic guided vehicle during driving, to obtain the historical moving speed data;

[0071] S103, using a camera installed on the automatic guided vehicle to scan a two-dimensional code on the driving path to obtain the historical vehicle head angle data; wherein, there are multiple two-dimensional codes, and the multiple two-dimensional codes are distributed in a matrix on the driving path.

[0072] S2. Using the historical motion data to train a deep reinforcement learning model to obtain a movement deviation prediction model; the deep reinforcement learning model is specifically a proximal strategy optimization model;

[0073] The historical motion data is used to train a deep reinforcement learning model to obtain a motion deviation prediction model, which specifically includes the following steps:

[0074] S201, constructing the deep reinforcement learning model based on the proximal strategy optimization algorithm to obtain an initial prediction model;

[0075] S202, using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to train the initial prediction model to obtain an initial strategy network model;

[0076] S203, using an inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving to obtain real-time acceleration data;

[0077] S204, using an odometer to measure the real-time moving speed of the automatic guided vehicle during driving, to obtain real-time moving speed data;

[0078] S205, using a camera installed on the automatic guided vehicle to scan the QR code on the driving path in real time to obtain real-time vehicle head angle data and real-time position deviation data;

[0079] S206, using the real-time acceleration data and the real-time moving speed data to input the initial strategy network model to predict initial position deviation prediction data and initial vehicle head angle prediction data of the automatic guided vehicle;

[0080] S207, comparing the difference between the real-time position deviation data and the initial position deviation prediction data to obtain a position prediction difference result;

[0081] S208, comparing the difference between the initial vehicle head angle prediction data and the real-time vehicle head angle data to obtain an angle prediction difference result;

[0082] S209, optimizing the initial strategy network model using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model.

[0083] The initial prediction model is trained using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to obtain an initial strategy network model, which specifically includes the following steps:

[0084] Splitting the historical acceleration data into acceleration training data and acceleration verification data;

[0085] Splitting the historical moving speed data into moving speed training data and moving speed verification data;

[0086] Using the acceleration training data, the moving speed training data and the historical vehicle head angle data to train the initial prediction model, to obtain the initial strategy network model;

[0087] The initial strategy network model is optimized by using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model, which specifically includes the following steps:

[0088] Calculate model optimization parameters according to the position prediction difference result and the angle prediction difference result;

[0089] The initial strategy network model is optimized using the model optimization parameters to obtain the movement deviation prediction model.

[0090] like Figure 2As shown in the figure, the evaluation network in the proximal policy optimization algorithm is used to evaluate the policy network old, that is, the above-mentioned initial policy network model, to obtain the policy gradient. The optimizer uses the policy gradient to update the parameters of the policy network, and uses the updated parameters to build a new policy network new. The parameters of the policy network new are adjusted to the initial policy network old to complete the policy network optimization. Figure 2 middle, represents the binary state of the policy network old, The reward value of the reward function corresponding to the policy network old, The average value of the action probability density function of the policy network old and variance , Represents the binary state of the policy network new.

[0091] S3, collecting real-time motion data of the automatic guided vehicle; the real-time motion data includes real-time acceleration data and the real-time moving speed data;

[0092] Collecting the real-time motion data of the automatic guided vehicle specifically includes the following steps:

[0093] S301, using the inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving, to obtain the real-time acceleration data;

[0094] S302: Using an odometer to measure the real-time moving speed of the automatic guided vehicle during driving, to obtain the real-time moving speed data.

[0095] S4, inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data;

[0096] The real-time motion data is input into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data, specifically:

[0097] The real-time acceleration data and the real-time moving speed data are input into the moving deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle, and obtain the inertial measurement speed deviation value, the mileage measurement speed deviation value and the angle deviation value; wherein the position deviation prediction data includes the inertial measurement speed deviation value, the mileage measurement speed deviation value and the angle deviation value.

[0098] S5. Calculate the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data to achieve positioning of the automatic guided vehicle.

[0099] Calculating the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data to achieve positioning of the automatic guided vehicle specifically includes the following steps:

[0100] S501, using a Kalman filter algorithm to calculate a displacement value of the automatic guided vehicle according to the real-time motion data, the inertial measurement speed deviation value, and the mileage measurement speed deviation value;

[0101] S502, calculating the head deviation angle of the automatic guided vehicle according to the angle deviation value;

[0102] S503, calculating the position of the automatic guided vehicle according to the displacement value and the head deviation angle to achieve positioning of the automatic guided vehicle. The head deviation angle is an absolute value of the head deviation, that is, a value when the head of the guided vehicle deviates from a preset track direction.

[0103] The Kalman filter algorithm is used to calculate the displacement value of the automatic guided vehicle according to the real-time motion data, the inertial measurement speed deviation value and the mileage measurement speed deviation value, which specifically includes the following steps:

[0104] The Kalman gain is calculated using the inertial measurement speed deviation value and the mileage measurement speed deviation value. The calculation formula for calculating the Kalman gain is as follows:

[0105] ;

[0106] in, represents the Kalman gain, Indicates the inertial measurement speed deviation value, Indicates the mileage measurement speed deviation value.

[0107] A Kalman filter algorithm is used to calculate the displacement value of the automatic guided vehicle according to the real-time motion data and the Kalman gain.

[0108] In the embodiment of the present invention, a continuous two-dimensional code matrix is ​​arranged on the ground, such as Figure 3As shown, the absolute position and angle of the automatic guided vehicle are obtained as precise values ​​by scanning the QR code in a cycle using a camera, and the speed deviation of the inertial measurement device and the wheel speed odometer at the current moment and the previous moment and the angle deviation of the real-time vehicle head direction are predicted by training using a proximal strategy optimization algorithm. The two fused estimated speeds are obtained by using a Kalman filter algorithm, and the estimated speed integral is used to obtain the estimated displacement. The precise displacement and angle deviation are calculated by combining the absolute position and angle on the QR code matrix, and the reward function of the proximal strategy optimization algorithm is designed according to the deviation as a basis for judging the quality of the training prediction parameters. The training model can also be used on QR codes with large spacing and straight lines. It can be seen that the present invention presets the position deviation data of the automatic guided vehicle by using a deep reinforcement learning model, and calculates the actual position of the guided vehicle by using the measured data and the position deviation data of the guided vehicle during the movement, that is, by compensating the real-time measurement data by the position deviation data, so as to improve the accuracy of the position calculation of the guided vehicle.

[0109] like Figure 4 As shown in the figure, even if the guide vehicle is traveling on a path with a large distance between two adjacent QR codes, its position can be accurately calculated, which improves the positioning accuracy of the guide vehicle. When the car is traveling on a path with a large distance between two QR codes and multiple QR codes are arranged in a straight line, the car can correct the path according to the predicted deviation, realizing accurate navigation of the car, i.e. the guide vehicle.

[0110] like Figure 5 As shown, in some embodiments, the method of using the Kalman filter algorithm to calculate the position of the automatic guided vehicle is as follows:

[0111] Use the Kalman filter algorithm to fuse velocity and estimate displacement:

[0112] Acceleration measured by an inertial measurement unit (IMU) during the operation of an AGV (Automated Guided Vehicle) and odometer speed , for the inertial measurement unit acceleration Points to get speed , for the inertial measurement unit velocity and odometer speed Perform Kalman filter fusion:

[0113] Kalman gain: ;

[0114] Inertial measurement unit and odometry fusion estimate speed: ;in, Indicated in Odometer speed at the time, Indicates at time speed, Indicates at time The Kalman gain, Indicates at time The estimated speed, Indicates at time Estimated speed.

[0115] Estimated displacement: ;in, Indicates at time The estimated displacement.

[0116] Proximal Policy Optimization Algorithm Estimates:

[0117] The velocity deviation of the inertial measurement device during the process of Kalman filter fusion velocity estimation displacement The speed deviation before and after the odometer It is difficult to calculate. Deep reinforcement learning training estimation has higher reliability and adaptability. The acceleration of the inertial measurement device is input into the proximal strategy optimization algorithm. and odometer speed , output the speed deviation of the inertial measurement device , the speed deviation of the odometer and angular deviation ;

[0118] The proximal policy optimization algorithm consists of two policy networks and one evaluation network. The network parameters are initialized and the acceleration of the inertial measurement device is and odometer speed As a binary state As the input of the policy network (old), generate the corresponding action , use this action in Kalman filter fusion to get the next state , the reward value can also be obtained through the reward function , and output the average value of the probability density function of the old action and variance ,Will It is stored in the experience replay pool as experience data for use in policy network (new) and evaluation network training. Represents the average value of the action probability density function and variance ; The purpose of the reward function design is to reduce the error between the estimated displacement and the actual displacement, and the difference between the vehicle head angle and the actual angle. The reward function is:

[0119] ;

[0120] is the standard displacement calculated for the QR code matrix, is the angle difference calculated in the QR code matrix, Represents the reward value, represents the displacement reward function, which outputs the displacement reward value; Represents the angle reward function, which outputs the angle reward value.

[0121] Compute the advantage function:

[0122] Replay the data generated in the experience pool In Input into the policy network new and the evaluation network, and output the reward for each action through the policy network old The value obtained by inputting the state into the evaluation network Calculate the advantage function and use the advantage function to evaluate the goodness of an action relative to other actions. The advantage function can be calculated using generalized advantage estimation, and the calculation process is:

[0123] ;

[0124] in, is the discount factor, Indicates at time The advantage estimate of the advantage function, Indicates at time The discount factor is Indicates time At the moment after, Indicates from time To time of The cumulative summation formula is: Indicates at time The reward value of . Usually, Between [0, 1], used to balance bias and variance.

[0125] Update policy network:

[0126] The loss function of the policy network is as follows:

[0127] ;

[0128] After the solution is completed, the policy gradient is obtained and the policy network parameters are updated, where: represents the policy gradient, Represents the updated policy network at time The advantage estimate of the advantage function, is the probability ratio of the strategy:

[0129] ;

[0130] It is a hyperparameter, usually between 0.1 and 0.3, used to control the magnitude of the policy update; represents the average value of the action probability density function of the policy network new, Represents the average value of the action probability density function of the policy network old. The role of the loss function is to update according to the normal advantage function if the policy changes slightly; if the change is large, it is limited to a range.

[0131] Update the evaluation network:

[0132] Value Network The update uses the standard mean squared error (MSE) loss function:

[0133] ;

[0134] in, From the state The discount accumulative reward starts from Representing the Value Network The mean square error, Indicates status The corresponding evaluation network value is Indicates status The corresponding evaluation network value is represents the mean square error function, represents the difference between two samples in the mean square error function, is a sample of the mean square error function, is another sample of the mean squared error function, estimated using the temporal difference method. By optimizing this loss function, the parameters of the value network are updated to better estimate the state value function.

[0135] Training model using:

[0136] When the training model reward gradually increases and approaches convergence, it means that the model estimation result is close to the accurate value measured on the QR code matrix. The reward function module is removed, and the training parameters of the network are loaded for use. The acceleration of the inertial measurement device and the odometer acceleration are input into the algorithm to estimate the accurate displacement and angle of the automatic guided vehicle, and realize navigation operation on the QR code path with large spacing and straight lines.

[0137] like Figure 6 As shown, in some other embodiments, an automatic guided vehicle positioning system based on deep reinforcement learning is also provided, including:

[0138] A data acquisition module, used to collect historical movement data of the automated guided vehicle;

[0139] A model building module, used to train a deep reinforcement learning model using the historical motion data to obtain a movement deviation prediction model;

[0140] The data acquisition module is also used to collect real-time motion data of the automatic guided vehicle;

[0141] A prediction processing module, used for inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data;

[0142] A position calculation module is used to calculate the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data, so as to realize the positioning of the automatic guided vehicle.

[0143] Among them, the data acquisition module, the model building module, the prediction processing module and the position calculation module can be program function modules or devices with corresponding functions or computers with corresponding functions.

[0144] Example 2

[0145] Based on Example 1, this embodiment provides an automatic guided vehicle positioning device, including a memory and one or more processors, wherein the memory stores executable code, and when one or more of the processors execute the executable code, the steps of the automatic guided vehicle positioning method based on deep reinforcement learning in Example 1 are implemented.

[0146] The memory may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The memory may also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart memory card, an SD card, a flash memory card, etc. equipped on the device. Furthermore, the memory may include both an internal storage unit and an external storage device of any device with data processing capability. The memory is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the concept and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for positioning an automatic guided vehicle based on deep reinforcement learning, characterized in that: The steps include: Collect historical movement data of automated guided vehicles; Using the historical motion data to train a deep reinforcement learning model to obtain a movement deviation prediction model; Collecting real-time motion data of the automatic guided vehicle; Inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data; Calculating the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data to achieve positioning of the automatic guided vehicle; The historical motion data includes historical acceleration data, historical moving speed data, and historical vehicle head angle data of the automated guided vehicle; Collect historical movement data of the AGV, including: Using an inertial measurement device to measure the acceleration of the automatic guided vehicle during driving, to obtain the historical acceleration data; Using an odometer to measure the moving speed of the automatic guided vehicle during driving, to obtain the historical moving speed data; The historical vehicle head angle data is obtained by scanning a two-dimensional code on a driving path using a camera installed on the automatic guided vehicle; wherein a plurality of two-dimensional codes are provided, and the plurality of two-dimensional codes are distributed in a matrix on the driving path; The deep reinforcement learning model is specifically a proximal strategy optimization model; The historical motion data is used to train a deep reinforcement learning model to obtain a motion deviation prediction model, which specifically includes the following steps: Constructing the deep reinforcement learning model based on the proximal strategy optimization algorithm to obtain an initial prediction model; Using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to train the initial prediction model to obtain an initial strategy network model; Using an inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving to obtain real-time acceleration data; Using an odometer to measure the real-time moving speed of the automatic guided vehicle during driving to obtain real-time moving speed data; Using a camera installed on the automatic guided vehicle to scan the QR code on the driving path in real time, to obtain real-time vehicle head angle data and real-time position deviation data; Using the real-time acceleration data and the real-time moving speed data to input the initial strategy network model to predict initial position deviation prediction data and initial vehicle head angle prediction data of the automatic guided vehicle; Comparing the difference between the real-time position deviation data and the initial position deviation prediction data to obtain a position prediction difference result; Comparing the difference between the initial vehicle head angle prediction data and the real-time vehicle head angle data to obtain an angle prediction difference result; The initial strategy network model is optimized using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model.

2. The automatic guided vehicle positioning method based on deep reinforcement learning according to claim 1 is characterized in that: The initial prediction model is trained using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to obtain an initial strategy network model, which specifically includes the following steps: Splitting the historical acceleration data into acceleration training data and acceleration verification data; Splitting the historical moving speed data into moving speed training data and moving speed verification data; Using the acceleration training data, the moving speed training data and the historical vehicle head angle data to train the initial prediction model, to obtain the initial strategy network model; The initial strategy network model is optimized by using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model, which specifically includes the following steps: Calculate model optimization parameters according to the position prediction difference result and the angle prediction difference result; The initial strategy network model is optimized using the model optimization parameters to obtain the movement deviation prediction model.

3. The automatic guided vehicle positioning method based on deep reinforcement learning according to claim 1 is characterized in that: The real-time motion data includes real-time acceleration data and real-time moving speed data; Collecting the real-time motion data of the automatic guided vehicle specifically includes the following steps: Using the inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving, to obtain the real-time acceleration data; The real-time moving speed of the automatic guided vehicle during driving is measured by using an odometer to obtain the real-time moving speed data.

4. The automatic guided vehicle positioning method based on deep reinforcement learning according to claim 3 is characterized in that: The real-time motion data is input into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data, specifically: The real-time acceleration data and the real-time moving speed data are input into the moving deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle, and obtain the inertial measurement speed deviation value, the mileage measurement speed deviation value and the angle deviation value; wherein the position deviation prediction data includes the inertial measurement speed deviation value, the mileage measurement speed deviation value and the angle deviation value.

5. The automatic guided vehicle positioning method based on deep reinforcement learning according to claim 4 is characterized in that: Calculating the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data to achieve positioning of the automatic guided vehicle specifically includes the following steps: Using a Kalman filter algorithm to calculate the displacement value of the automatic guided vehicle according to the real-time motion data, the inertial measurement speed deviation value, and the mileage measurement speed deviation value; Calculating the head deviation angle of the automatic guided vehicle according to the angle deviation value; The position of the automatic guided vehicle is calculated according to the displacement value and the vehicle head offset angle to achieve positioning of the automatic guided vehicle.

6. The automatic guided vehicle positioning method based on deep reinforcement learning according to claim 5 is characterized in that: The Kalman filter algorithm is used to calculate the displacement value of the automatic guided vehicle according to the real-time motion data, the inertial measurement speed deviation value and the mileage measurement speed deviation value, which specifically includes the following steps: Calculating a Kalman gain using the inertial measurement speed deviation value and the mileage measurement speed deviation value; A Kalman filter algorithm is used to calculate the displacement value of the automatic guided vehicle according to the real-time motion data and the Kalman gain.

7. An automatic guided vehicle positioning system based on deep reinforcement learning, characterized in that: include: A data acquisition module, used to collect historical movement data of the automated guided vehicle; A model building module, used to train a deep reinforcement learning model using the historical motion data to obtain a movement deviation prediction model; The data acquisition module is also used to collect real-time motion data of the automatic guided vehicle; A prediction processing module, used for inputting the real-time motion data into the movement deviation prediction model to predict the position deviation and the head angle of the automatic guided vehicle to obtain position deviation prediction data; A position calculation module, used to calculate the position of the automatic guided vehicle according to the real-time motion data and the position deviation prediction data, so as to realize the positioning of the automatic guided vehicle; The historical motion data includes historical acceleration data, historical moving speed data, and historical vehicle head angle data of the automated guided vehicle; Collect historical movement data of the AGV, including: Using an inertial measurement device to measure the acceleration of the automatic guided vehicle during driving, to obtain the historical acceleration data; Using an odometer to measure the moving speed of the automatic guided vehicle during driving, to obtain the historical moving speed data; The historical vehicle head angle data is obtained by scanning a two-dimensional code on a driving path using a camera installed on the automatic guided vehicle; wherein a plurality of two-dimensional codes are provided, and the plurality of two-dimensional codes are distributed in a matrix on the driving path; The deep reinforcement learning model is specifically a proximal strategy optimization model; The historical motion data is used to train a deep reinforcement learning model to obtain a motion deviation prediction model, which specifically includes the following steps: Constructing the deep reinforcement learning model based on the proximal strategy optimization algorithm to obtain an initial prediction model; Using the historical acceleration data, the historical moving speed data, and the historical vehicle head angle data to train the initial prediction model to obtain an initial strategy network model; Using an inertial measurement device to measure the real-time acceleration of the automatic guided vehicle during driving to obtain real-time acceleration data; Using an odometer to measure the real-time moving speed of the automatic guided vehicle during driving to obtain real-time moving speed data; Using a camera installed on the automatic guided vehicle to scan the QR code on the driving path in real time, to obtain real-time vehicle head angle data and real-time position deviation data; Using the real-time acceleration data and the real-time moving speed data to input the initial strategy network model to predict initial position deviation prediction data and initial vehicle head angle prediction data of the automatic guided vehicle; Comparing the difference between the real-time position deviation data and the initial position deviation prediction data to obtain a position prediction difference result; Comparing the difference between the initial vehicle head angle prediction data and the real-time vehicle head angle data to obtain an angle prediction difference result; The initial strategy network model is optimized using the position prediction difference result and the angle prediction difference result to obtain the movement deviation prediction model.

8. An automatic guided vehicle positioning device, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when one or more of the processors execute the executable code, the steps of the automatic guided vehicle positioning method based on deep reinforcement learning as described in any one of claims 1 to 6 are implemented.

Images