Unmanned platform track prediction method based on reinforcement learning and improved potential field method

Abstract

The invention belongs to the technical field of unmanned platform track planning, and particularly discloses an unmanned platform track prediction method based on reinforcement learning and an improved potential field method, and the method comprises the following steps: obtaining track parameters of an unmanned platform in a continuous time period as sample data; fitting a historical track of the unmanned platform by using a function fitting method based on the sample data; based on the static obstacle information and the dynamic obstacle information in the application scene, constructing a corresponding repulsive force field; initializing empirical playback algorithm parameters, setting a reward and punishment function based on the historical track, the static obstacle information, the dynamic obstacle information and the repulsive force field, and training a network model by using the reward and punishment function; and utilizing the trained network model to predict the track of the unmanned platform in a period of time in the future. According to the invention, the training efficiency of the unmanned platform track prediction model and the accuracy of the prediction result can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of unmanned platform trajectory planning, and more specifically, relates to an unmanned platform trajectory prediction method based on reinforcement learning and improved potential field method. Background Art

[0002] With the continuous development of artificial neural networks, its related technologies have been applied to trajectory prediction. When solving complex nonlinear trajectory prediction problems based on neural networks, the target historical trajectory data and current trajectory data are used as input variables, and the future trajectory information is used as the output variable. By training the input variables and the true values, a function mapping relationship is established to predict the trajectory. However, an inappropriate neural network structure will have problems such as overfitting or non-convergence. Deep learning evolved from neural networks, and the learning of historical trajectory data features is more accurate, which makes up for the shortcomings of low prediction accuracy of traditional probability statistics methods. In deep learning, multi-layer perception (MLP) is widely used as the most basic network structure, but the existence of potential interference information can easily lead to poor prediction accuracy. Recurrent neural network (RNN) can capture time characteristics and has short-term memory function, but when RNN predicts longer trajectory data, improper parameter selection will cause gradient disappearance or explosion, resulting in large errors in the prediction results. When the traditional artificial potential field method is applied to complex and unknown environments, it will cause pathfinding failure due to reasons such as unreachable targets or local extreme points. Deep reinforcement learning allows the intelligent agent to learn relevant strategies for avoiding obstacles and finding target points in the process of constantly "making mistakes", and finally obtains an optimal path, but the overall training efficiency is low. In summary, with the popularization of intelligent transportation and smart cities, a large amount of trajectory data has provided development momentum for the research on trajectory prediction of mobile targets based on deep learning, and has also achieved good prediction results. However, the current methods still have some problems that need to be solved: 1. The unmanned platform trajectory prediction method based on deep learning or reinforcement learning has a slow training speed; 2. The prediction effect depends on the quality of model training. When the actual situation is complex and changeable, the accuracy and timeliness need to be further improved; 3. The existing unmanned platform track prediction algorithms are respectively for drones, unmanned vehicles, unmanned ships and other equipment, and no unified motion characteristics are extracted. Summary of the invention

[0003] In view of the defects of the prior art, the purpose of this application is to provide an unmanned platform trajectory prediction method based on reinforcement learning and improved potential field method, aiming to solve the problems of poor efficiency and low accuracy of existing unmanned platform trajectory prediction methods.

[0004] To achieve the above objectives, according to the first aspect of the present application, the present application provides an unmanned platform track prediction method based on reinforcement learning and improved potential field method, comprising:

[0005] S1 obtains the track parameters of the unmanned platform in a continuous time period as sample data;

[0006] S2 fits the historical track of the unmanned platform using a function fitting method based on the sample data;

[0007] S3 constructs the corresponding repulsive field based on the static obstacle information and dynamic obstacle information in the application scenario;

[0008] S4 initializes the experience playback algorithm parameters, and sets a reward and punishment function based on the historical track, static obstacle information, dynamic obstacle information and the repulsive field, and trains the network model using the reward and punishment function;

[0009] S5 uses the network model trained in step S4 to predict the trajectory of the unmanned platform in the future.

[0010] This application uses reinforcement learning to train an intelligent agent to explore autonomously in an uncertain environment, and uses an improved potential field method to accelerate the learning rate of the intelligent agent, form a prediction of the future trajectory of the unmanned platform, and assist in manual control of the current situation.

[0011] Furthermore, in step S2, the method of fitting the historical track using a function fitting method includes the following steps:

[0012] S201: taking the area between every two adjacent track points as an interpolation area, and interpolating in the interpolation area to supplement the track;

[0013] S202: dividing the interpolation area after the supplemented track into a plurality of sub-intervals, and constructing a corresponding linear polynomial using any two adjacent interpolation points in the sub-intervals; wherein the values ​​of the interpolation functions on the left and right sides of the division points of adjacent sub-intervals are equal;

[0014] S203 uses the linear polynomial to calculate the function value corresponding to the non-interpolation point in the corresponding sub-interval to obtain a piecewise function;

[0015] S204 compares the actual track point data to verify the accuracy of the piecewise function to obtain the historical track.

[0016] Furthermore, the linear polynomial is:

[0017]

[0018] Among them, x is the interpolation point, x i is the i-th interpolation point, x i+1is the i+1th interpolation point, y i is x i The corresponding function value, y i+1 is x i+1 The function value corresponding to the interpolation point.

[0019] Furthermore, in step S3, the repulsive field is constructed using the following formula:

[0020]

[0021] Among them, F is the magnitude of the repulsive force, gain rp is the repulsive force constant, dis po is the actual distance between the unmanned platform and the center of mass of a static obstacle or the center of mass of a dynamic obstacle. imp is the influence range of static obstacles or dynamic obstacles, area fb It is the prohibited range of static obstacles or dynamic obstacles.

[0022] Furthermore, in step S1, the track parameters include longitude, latitude, heading, speed and pitch angle. The purpose of this setting is to unify the motion characteristics of unmanned platforms such as drones and unmanned ships, so that the predicted track can be applied to more complex application scenarios.

[0023] Furthermore, the reward and punishment function is:

[0024]

[0025] Where F is the magnitude of the repulsive force, dis l is the distance between the current track point of the unmanned platform and the fitted route, pa It is the actual distance between the unmanned platform and the obstacle boundary.

[0026] Furthermore, the method for training a network model using the experience replay algorithm comprises the following steps:

[0027] S401 In each time step, the unmanned platform selects and executes actions based on the greedy strategy according to the current value table; the current environment gives rewards or penalties based on the reward and punishment function, and the unmanned platform observes the current environment and stores the current state, executed actions, next state, and rewards or penalties obtained as experience data in the experience playback buffer;

[0028] S402 randomly extracts a batch of experience data from the experience playback buffer to update the Q network of the unmanned platform; uses the mean square error loss and gradient descent method to update the Q network parameters, and copies the Q network parameters to the Q′ network at regular step lengths, where the update formula is:

[0029]

[0030] Where L(θ) is the loss function; maxQ'(s',a') is the maximum Q value of all possible actions in the next state (s'); s' is the state reached after executing action aˊ; θ is the Q network parameter, s is the state reached after executing action a; r is the immediate reward, γ is the discount factor used to balance the immediate reward and future rewards; E is the expectation;

[0031] S403 repeats steps S401 to S402, iteratively updates the target network, and stops training when the prediction accuracy of the trained network model meets the preset requirements.

[0032] In a second aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0033] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0034] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0035] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0036] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0037] 1) This application introduces an experience replay mechanism and an improved potential field method, which improves the efficiency and stability of model learning, while reducing dependence on the model, reducing the difficulty and cost of modeling; through reinforcement learning, the intelligent agent is trained to explore autonomously in an uncertain environment, supplemented by the improved potential field method to accelerate the learning rate of the intelligent agent, and form an accurate prediction of the future trajectory of the unmanned platform.

[0038] 2) During the model training phase, the trajectory prediction method of the present application comprehensively considers the influence of the unmanned platform's motion law, environment, and obstacle information in the environment on the unmanned platform's motion, thereby improving the accuracy of the unmanned platform's trajectory prediction.

[0039] 3) In addition to the traditional three-dimensional coordinates, this application also extracts the heading and pitch angle of the unmanned platform as parameters for track prediction, and uniformly integrates the motion characteristics of unmanned platforms such as drones and unmanned ships, so that the predicted unmanned platform track can truly cover the entire environmental space. It has strong adaptability, can handle higher-dimensional state and action spaces, and is suitable for more complex application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the unmanned platform track prediction method based on reinforcement learning and improved potential field method provided by this application;

[0041] Figure 2 It is a flow chart of the method for fitting historical tracks provided by this application;

[0042] Figure 3 It is a method flow chart of the experience replay technology training model provided by this application;

[0043] Figure 4 It is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0046] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0047] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0048] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0049] Next, the technical solutions provided in the embodiments of the present application are introduced.

[0050] This embodiment provides an unmanned platform track prediction method based on reinforcement learning and improved potential field method. Figure 1 As shown, the following steps are included:

[0051] S1 constructs sample data: obtains the track parameters of the unmanned platform in a continuous time period as sample data; the state of the unmanned platform is composed of parameters such as longitude, latitude, heading, speed, and pitch angle, so the track can be understood as a set of states arranged in chronological order.

[0052] S2 Historical track fitting: Based on sample data, the historical track of the unmanned platform is fitted using the function fitting method; specifically, a function is fitted to the existing track of the unmanned platform, and the function curve is used as one of the important indicators for future track prediction.

[0053] In this embodiment, since the track of the unmanned platform is a smooth curve most of the time, it can be represented by a linear or quadratic function, but there are also turning points. There is a large difference in the track before and after the turning point, which cannot be represented by the same function. Therefore, this method uses a piecewise function method to fit the historical track of the unmanned platform. The linear interpolation method can be used to directly and quickly add more coordinate points between the track points, so that the piecewise function can be used to fit the track more accurately.

[0054] Specifically, the piecewise function fitting method is used to fit the existing track of the unmanned platform, and the curve of the function is used as one of the objectives of reinforcement learning. The reward function is set accordingly, that is, the closer the predicted future track point of the unmanned platform is to the curve of the function, the higher the reward score is, such as Figure 2 As shown, the fitting method includes the following steps:

[0055] S201 determines the interpolation points: in this embodiment, the interpolation points are the existing track point observation data, and the area between every two adjacent track points is used as the interpolation area, and the points inserted later in the interpolation area are used as the track supplement, so as to perform track fitting and obtain the historical track;

[0056] S202 constructs interpolation polynomial: divides the interpolation interval into multiple subintervals [x i ,x i+1 ], and use any two adjacent interpolation points (x i ,y i )、(xi+1 ,y i+1 ) construct the corresponding linear polynomial;

[0057] The constructed linear polynomial is:

[0058]

[0059] Among them, x is the interpolation point, x i is the i-th interpolation point, x i+1 is the i+1th interpolation point, y i is x i The corresponding function value, y i+1 is x i+1 The function value corresponding to the interpolation point; the values ​​of the interpolation function on both sides of the division point of adjacent subintervals are equal, that is, P i (x i+1 )=P i+1 (x i+1 ).

[0060] S203 uses the linear polynomial (1) to calculate the function value corresponding to the non-interpolation point in the corresponding sub-interval to obtain the interpolation function;

[0061] S204 compares the actual data to verify the accuracy of the interpolation function; specifically, by comparing with the actual track point data, the accuracy and effect of the interpolation function are verified, and a piecewise function constructed by a linear interpolation method (i.e., a historical track fitting function) is obtained, which can be used to approximate the original function.

[0062] S3 constructs scene parameters: Based on the static obstacle information and dynamic obstacle information in the application scenario, a corresponding repulsion field is constructed; specifically, the static obstacle information and dynamic obstacle information in the input scene are formed for each obstacle, and each of the above obstacles and the corresponding repulsion field information are used as one of the important indicators for future track prediction.

[0063] Making the predicted track away from obstacles is also one of the goals of the track prediction algorithm. Therefore, a penalty function is set, that is, if the predicted future track point of the unmanned platform falls within the obstacle threat radius, a certain reward point will be deducted. When predicting the next track point of the unmanned platform, the influence of the repulsive field is considered first, and the track prediction point with smaller repulsive force is given priority.

[0064] In step S3, the repulsive field is constructed using the first formula:

[0065]

[0066] Among them, F is the magnitude of the repulsive force, gain rp is the repulsive force constant, dis pois the actual distance between the unmanned platform and the center of mass of a static obstacle or the center of mass of a dynamic obstacle. imp is the influence range of static obstacles or dynamic obstacles, area fb It is the prohibited range of static obstacles or dynamic obstacles.

[0067] S4 algorithm parameter initialization and model training: Initialize the algorithm, and set the reward and punishment function based on the historical track, static obstacle information, dynamic obstacle information and repulsion field information, and then use the reward and punishment function to train the network model; specifically, the initialization algorithm includes initializing the environment, setting the initial state, initializing the unmanned platform parameters, initializing the playback memory, etc. At the same time, the reward and punishment function is set based on the acquired historical track fitting function, dynamic obstacle information and static obstacle information.

[0068] 1) Initialize the environment

[0069] The environment defines the state and possible actions of the system, as well as the rules for how these actions change the state. In this embodiment, the environment is represented by (time range, longitude range, latitude range, altitude range), that is, the environment env = [[0, -180, -90, -1000], [1000, 180, 90, 1000]], and the action space act = [[-1], [1]].

[0070] 2) Set the initial state

[0071] Set the initial state of the unmanned platform, including timestamp, longitude, latitude, altitude, heading, speed, pitch angle, etc., where the heading is 0 degrees in the north direction and increases in the clockwise direction, the pitch angle is the angle between the head direction of the unmanned platform and the horizontal plane of the ground coordinate system, and the platform speed is uniformly in m / s. In this embodiment, the initial state fs = [0, 120, 30, 0, 120, 10, 0] is set.

[0072] 3) Initialize unmanned platform parameters

[0073] The unmanned platform parameters include the maximum speed, cruising range, maximum steering angle, etc. of the unmanned platform.

[0074] 4) Set up reward and punishment functions

[0075] The closer the track point of the unmanned platform fits the fitted track curve, the higher the score. The distance between the unmanned platform and the track curve ranges from [0, +∞]; calculate the repulsion F of the obstacle on the unmanned platform. The larger the repulsion F, the less reward. In the process of training and learning the agent, the size of the repulsion needs to be considered to determine the location of the next path point. At the same time, if the track point of the unmanned platform falls within the threat range of the obstacle, it will be punished by taking the distance between the track point and the center of mass of the obstacle as dis po , minus the distance from the intersection of the track point and the obstacle boundary to the center of the obstacle, the resulting value is the actual distance dis between the unmanned platform and the obstacle boundary pa , will be dis pa As one of the parameters of the reward and punishment function, namely, dis pa When it is a negative number, the larger the absolute value, the more severe the penalty. pa When it is a positive number, the larger the absolute value, the greater the reward.

[0076] The reward and punishment function is set as:

[0077]

[0078] Where F is the magnitude of the repulsive force, dis l is the distance between the current track point of the unmanned platform and the fitted route, pa It is the actual distance between the unmanned platform and the obstacle boundary.

[0079] 5) Model training

[0080] The network model is trained using the experience replay mechanism until the prediction accuracy of the trained network model meets the preset requirements and the training is stopped. The strategy network selected in this embodiment is the Q network.

[0081] like Figure 3 As shown, the method for training the network model using the aforementioned experience replay algorithm includes the following steps:

[0082] S401 In each time step, the unmanned platform selects and executes an action based on the greedy strategy according to the current value table; then the current environment gives a reward or punishment based on the reward and punishment function, and gives information on whether it is over; the unmanned platform observes the current environment and stores the current state, the executed action, the next state and the reward or punishment obtained as experience data in the experience replay buffer; specifically, the experience replay buffer is a storage structure (such as an array or a queue) created to store experience tuples, each of which contains the state s at time t t 、Action a t and reward r t , and the new state s at time t+1t+1 , each experience tuple is stored as a four-tuple (s t ,a t ,r t ,s t+1 ) and adds it to the experience replay buffer.

[0083] S402 When there is enough data in the experience replay buffer, a small batch of samples is randomly selected for training. These samples are used to calculate the loss function and update the parameters of the Q network. The purpose of random sampling is to break the correlation between the data and make the training process more stable.

[0084] Specifically, a batch of experience data is randomly extracted from the experience playback buffer to update the Q network parameters of the unmanned platform; the Q network parameters are updated using the mean square error loss and gradient descent method, and the Q network parameters are copied to the Q′ network at regular step lengths. The update formula is:

[0085]

[0086] Where L(θ) is the loss function; maxQ'(s′,a′) is the maximum Q value of all possible actions in the next state (s'); s′ is the state reached after executing action aˊ; θ is the Q network parameter, s is the state reached after executing action a; r is the immediate reward, γ is the discount factor used to balance the immediate reward and future reward; E is the expectation.

[0087] S403 repeats steps S401 to S402, iteratively updates the target network, and stops training when the prediction accuracy of the trained network model meets the preset requirements.

[0088] Specifically, the above process is repeated to allow the unmanned platform to continue learning and exploring in the environment. After a certain period of time, actions are selected according to the strategy, and the value function or strategy is updated using the data obtained from the exploration. When the predicted trajectory meets the accuracy requirements, the training process ends.

[0089] S5 track prediction

[0090] The model trained in step S4 is used to predict the track of the unmanned platform in the future, that is, based on the regular experience learned by the model, the track of the unmanned platform in the future, such as the next 60 seconds, is predicted.

[0091] This embodiment introduces an experience replay mechanism and an improved potential field method, which improves learning efficiency and stability, while reducing dependence on models, reducing the difficulty and cost of modeling; it also uniformly integrates the motion characteristics (such as heading and pitch angle) of various unmanned platforms such as drones and unmanned ships, so that the predicted unmanned platform track can truly cover the entire environmental space, has strong adaptability, can handle higher-dimensional state and action spaces, and is suitable for more complex application scenarios.

[0092] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device, which may include: a processor 110, a communication interface 120, a memory 130 and a communication bus 140, wherein the processor 110, the communication interface 120 and the memory 130 communicate with each other through the communication bus 140. The processor 110 may call the logic instructions in the memory 130 to execute the method in the above embodiment.

[0093] In addition, the logic instructions in the above-mentioned memory 130 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0094] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0095] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0096] It is understandable that the processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0097] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0098] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0099] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0100] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for unmanned platform track prediction based on reinforcement learning and improved potential field method, characterized in that: include: S1 obtains the track parameters of the unmanned platform in a continuous time period as sample data; S2 fits the historical track of the unmanned platform using a function fitting method based on the sample data; S3 constructs the corresponding repulsive field based on the static obstacle information and dynamic obstacle information in the application scenario; S4 initializes the experience playback algorithm parameters, and sets a reward and punishment function based on the historical track, static obstacle information, dynamic obstacle information and the repulsive field, and trains the network model using the reward and punishment function; S5 uses the network model trained in step S4 to predict the trajectory of the unmanned platform in the future.

2. The unmanned platform track prediction method based on reinforcement learning and improved potential field method as claimed in claim 1, characterized in that: In step S2, the method of fitting the historical track using a function fitting method comprises the following steps: S201: taking the area between every two adjacent track points as an interpolation area, and interpolating in the interpolation area to supplement the track; S202: dividing the interpolation area after the supplemented track into a plurality of sub-intervals, and constructing a corresponding linear polynomial using any two adjacent interpolation points in the sub-intervals; wherein the values ​​of the interpolation functions on the left and right sides of the division points of adjacent sub-intervals are equal; S203 uses the linear polynomial to calculate the function value corresponding to the non-interpolation point in the corresponding sub-interval to obtain a piecewise function; S204 compares the actual track point data to verify the accuracy of the piecewise function to obtain the historical track.

3. The unmanned platform track prediction method based on reinforcement learning and improved potential field method as claimed in claim 2, characterized in that: The linear polynomial is: Among them, x is the interpolation point, x i is the i-th interpolation point, x i+1 is the i+1th interpolation point, y i is x i The corresponding function value, y i+1 is x i+1 The function value corresponding to the interpolation point.

4. The unmanned platform track prediction method based on reinforcement learning and improved potential field method as claimed in claim 1, characterized in that: In step S3, the repulsive field is constructed using the following formula: Among them, F is the magnitude of the repulsive force, gain rp is the repulsive force constant, dis po is the actual distance between the unmanned platform and the center of mass of a static obstacle or the center of mass of a dynamic obstacle. imp is the influence range of static obstacles or dynamic obstacles, area fb It is the prohibited range of static obstacles or dynamic obstacles.

5. The unmanned platform track prediction method based on reinforcement learning and improved potential field method as claimed in claim 1, characterized in that: In step S1, the track parameters include longitude, latitude, heading, speed and pitch angle.

6. The unmanned platform track prediction method based on reinforcement learning and improved potential field method as claimed in claim 1, characterized in that: The reward and punishment function is: Where F is the magnitude of the repulsive force, dis l is the distance between the current track point of the unmanned platform and the fitted route, pa It is the actual distance between the unmanned platform and the obstacle boundary.

7. The unmanned platform track prediction method based on reinforcement learning and improved potential field method as claimed in claim 1, wherein the method of training the network model using the experience replay algorithm comprises the following steps: S401 In each time step, the unmanned platform selects and executes actions based on the greedy strategy according to the current value table; the current environment gives rewards or penalties based on the reward and punishment function, and the unmanned platform observes the current environment and stores the current state, executed actions, next state, and rewards or penalties obtained as experience data in the experience playback buffer; S402 randomly extracts a batch of experience data from the experience playback buffer to update the Q network of the unmanned platform; uses the mean square error loss and gradient descent method to update the Q network parameters, and copies the Q network parameters to the Q′ network at regular step lengths, where the update formula is: Where L(θ) is the loss function; maxQ'(s',a') is the maximum Q value of all possible actions in the next state (s'); s' is the state reached after executing action aˊ; θ is the Q network parameter, s is the state reached after executing action a; r is the immediate reward, γ is the discount factor used to balance the immediate reward and future rewards; E is the expectation; S403 repeats steps S401 to S402, iteratively updates the target network, and stops training when the prediction accuracy of the trained network model meets the preset requirements.

8. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images