Unmanned platform path prediction method based on reinforcement learning and improved potential field method

Abstract

The application belongs to the technical field of unmanned platform path planning, and specifically discloses an unmanned platform path prediction method based on reinforcement learning and improved potential field method, which comprises the following steps: acquiring path parameters of the unmanned platform in a continuous time period as sample data; fitting the historical path of the unmanned platform by using a function fitting method based on the sample data; constructing a corresponding repulsive field based on static obstacle information and dynamic obstacle information in an application scenario; initializing experience replay algorithm parameters, setting a reward and punishment function based on the historical path, the static obstacle information, the dynamic obstacle information and the repulsive field, and training a network model by using the reward and punishment function; and predicting the path of the unmanned platform in a future period of time by using the trained network model. The application can improve the training efficiency of the unmanned platform path prediction model and the accuracy of the prediction result.

Description

Technical Field

[0001] The present application belongs to the technical field of unmanned platform trajectory planning, and more specifically, relates to an unmanned platform trajectory prediction method based on reinforcement learning and improved potential field method. Background Art

[0002] With the continuous development of artificial neural networks, related technologies have been applied to trajectory prediction. When solving complex, nonlinear trajectory prediction problems based on neural networks, historical and current target trajectory data are used as input variables, and future trajectory information is used as output variables. Trajectory prediction is achieved by training the input variables with true values ​​and establishing a function mapping relationship. However, inappropriate neural network structures can lead to problems such as overfitting and non-convergence. Deep learning, which evolved from neural networks, more accurately learns the characteristics of historical trajectory data, addressing the low prediction accuracy of traditional probabilistic and statistical methods. Multi-layer perceptrons (MLPs) are widely used as the most basic network structure in deep learning, but the presence of potential interference can lead to poor prediction accuracy. Recurrent neural networks (RNNs) can capture temporal features and have short-term memory. However, when predicting longer trajectory data, improper parameter selection can cause vanishing or exploding gradients, resulting in large prediction errors. Traditional artificial potential field methods, when applied to complex, unknown environments, can fail in pathfinding due to unreachable targets or local extreme points. Deep reinforcement learning, on the other hand, allows intelligent agents to learn strategies for avoiding obstacles and finding target points through continuous "errors," ultimately achieving an optimal path. However, the overall training efficiency is relatively low. In summary, with the widespread adoption of intelligent transportation and smart cities, a large amount of trajectory data has provided impetus for the development of mobile target trajectory prediction research based on deep learning, and has also achieved good prediction results. However, current methods still have some urgent problems that need to be solved: 1. Unmanned platform trajectory prediction methods based on deep learning or reinforcement learning are slow to train; 2. Prediction results depend on the quality of model training. When the actual situation is complex and changeable, the accuracy and timeliness need to be further improved; 3. Existing unmanned platform track prediction algorithms are tailored to drones, unmanned vehicles, unmanned ships, and other equipment, without refining unified motion characteristics. Summary of the Invention

[0003] In response to the defects of the existing technology, the purpose of this application is to provide an unmanned platform trajectory prediction method based on reinforcement learning and improved potential field method, aiming to solve the problems of poor efficiency and low accuracy of existing unmanned platform trajectory prediction methods.

[0004] To achieve the above objectives, according to the first aspect of the present application, the present application provides an unmanned platform track prediction method based on reinforcement learning and improved potential field method, comprising:

[0005] S1 obtains the track parameters of the unmanned platform in a continuous time period as sample data;

[0006] S2 fits the historical track of the unmanned platform using a function fitting method based on the sample data;

[0007] S3 constructs the corresponding repulsive field based on the static obstacle information and dynamic obstacle information in the application scenario;

[0008] S4 initializes experience replay algorithm parameters, sets a reward and punishment function based on the historical track, static obstacle information, dynamic obstacle information, and the repulsive field, and trains the network model using the reward and punishment function;

[0009] S5 uses the network model trained in step S4 to predict the trajectory of the unmanned platform in the future.

[0010] This application uses reinforcement learning to train an intelligent agent to autonomously explore in an uncertain environment, supplemented by an improved potential field method to accelerate the learning rate of the intelligent agent, form a prediction of the future trajectory of the unmanned platform, and assist humans in controlling the current situation.

[0011] Furthermore, in step S2, the method of fitting the historical track using a function fitting method includes the following steps:

[0012] S201: taking the area between every two adjacent track points as an interpolation area, and interpolating in the interpolation area to supplement the track;

[0013] S202: dividing the interpolation area after the supplemented track into a plurality of subintervals, and constructing corresponding linear polynomials using any two adjacent interpolation points in the subintervals; wherein the values ​​of the interpolation functions on the left and right sides of the division points of adjacent subintervals are equal;

[0014] S203 uses the linear polynomial to calculate the function value corresponding to the non-interpolation point in the corresponding subinterval to obtain a piecewise function;

[0015] S204 compares the actual track point data to verify the accuracy of the piecewise function to obtain the historical track.

[0016] Furthermore, the linear polynomial is:

[0017]

[0018] Among them, x is the interpolation point, x i is the i-th interpolation point, x i+1is the i+1th interpolation point, y i is x i The corresponding function value, y i+1 is x i+1 The function value corresponding to the interpolation point.

[0019] Furthermore, in step S3, the repulsive field is constructed using the following formula:

[0020]

[0021] Among them, F is the magnitude of the repulsive force, gain rp is the repulsive force constant, dis po is the actual distance between the unmanned platform and the center of mass of a static obstacle or the center of mass of a dynamic obstacle. imp Is the range of influence of static obstacles or dynamic obstacles, area fb It is the prohibited range of static obstacles or dynamic obstacles.

[0022] Furthermore, in step S1, the track parameters include longitude, latitude, heading, speed, and pitch angle. The purpose of this setting is to unify and integrate the motion characteristics of unmanned platforms such as drones and unmanned ships, so that the predicted track can be applied to more complex application scenarios.

[0023] Furthermore, the reward and punishment function is:

[0024]

[0025] Among them, F is the magnitude of the repulsive force, dis l is the distance between the current track point of the unmanned platform and the fitted route, dis pa It is the actual distance between the unmanned platform and the obstacle boundary.

[0026] Furthermore, the method for training a network model using the experience replay algorithm includes the following steps:

[0027] S401: In each time step, the unmanned platform selects and executes an action based on the greedy strategy according to the current value table. The current environment gives rewards or penalties based on the reward and penalty function. The unmanned platform observes the current environment and stores the current state, executed action, next state, and reward or penalty obtained as experience data in the experience replay buffer.

[0028] S402 randomly extracts a batch of experience data from the experience replay buffer to update the Q network of the unmanned platform; uses the mean square error loss and gradient descent method to update the Q network parameters, and copies the Q network parameters to the Q′ network at regular step lengths, where the update formula is:

[0029]

[0030] Where L(θ) is the loss function; maxQ'(s',a') is the maximum Q value of all possible actions in the next state (s'); s' is the state reached after performing action a'; θ is the Q network parameter, s is the state reached after performing action a; r is the immediate reward, γ is the discount factor used to balance the immediate reward and future rewards; E is the expectation;

[0031] S403 repeats steps S401 to S402, iteratively updates the target network, and stops training when the prediction accuracy of the trained network model meets the preset requirements.

[0032] In a second aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0033] In a third aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0034] In a fourth aspect, the present application provides a computer program product, which, when executed on a processor, enables the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0035] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0036] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the existing technologies:

[0037] 1) This application introduces an experience replay mechanism and an improved potential field method, which improves the efficiency and stability of model learning, while reducing dependence on the model and lowering the difficulty and cost of modeling; through reinforcement learning, the intelligent agent is trained to explore autonomously in an uncertain environment, supplemented by the improved potential field method to accelerate the learning rate of the intelligent agent and form an accurate prediction of the future trajectory of the unmanned platform.

[0038] 2) During the model training phase, the trajectory prediction method of this application comprehensively considers the influence of the unmanned platform's motion patterns, the environment, and the obstacle information in the environment on the unmanned platform's motion, thereby improving the accuracy of the unmanned platform's trajectory prediction.

[0039] 3) In addition to traditional three-dimensional coordinates, this application also extracts the heading and pitch angle of the unmanned platform as parameters for track prediction, and uniformly integrates the motion characteristics of unmanned platforms such as drones and unmanned ships, so that the predicted unmanned platform track can truly cover the entire environmental space. It has strong adaptability and can handle higher-dimensional state and action spaces, making it suitable for more complex application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the unmanned platform trajectory prediction method based on reinforcement learning and improved potential field method provided by this application;

[0041] Figure 2 This is a flow chart of the method for fitting historical tracks provided by this application;

[0042] Figure 3 This is a flow chart of the method for training the experience replay technology model provided by this application;

[0043] Figure 4 This is a schematic diagram of the electronic device structure provided by this application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0045] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0046] The terms "first" and "second" in this specification and claims are used to distinguish different objects rather than to describe a specific order of objects. For example, "first response message" and "second response message" are used to distinguish different response messages rather than to describe a specific order of response messages.

[0047] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0048] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc.

[0049] Next, the technical solutions provided in the embodiments of this application are introduced.

[0050] This embodiment provides a method for unmanned platform trajectory prediction based on reinforcement learning and improved potential field method. Figure 1 As shown, the following steps are included:

[0051] S1 constructs sample data: obtains the track parameters of the unmanned platform in a continuous time period as sample data; the state of the unmanned platform is composed of parameters such as longitude, latitude, heading, speed, and pitch angle, so the track can be understood as a set of states arranged in chronological order.

[0052] S2 Historical Track Fitting: Based on sample data, the historical track of the unmanned platform is fitted using the function fitting method. Specifically, a function is fitted to the existing track of the unmanned platform, and the function curve is used as one of the important indicators for future track prediction.

[0053] In this embodiment, since the track of the unmanned platform is a smooth curve most of the time, it can be represented by a linear or quadratic function. However, there are also turning points. The track before and after the turning point is quite different and cannot be represented by the same function. Therefore, this method uses a piecewise function method to fit the historical track of the unmanned platform. The linear interpolation method can be used to directly and quickly add more coordinate points between the track points, so that the piecewise function can be used to fit the track segment more accurately.

[0054] Specifically, the piecewise function fitting method is used to fit the existing track of the unmanned platform, and the curve of the function is used as one of the objectives of reinforcement learning. The reward function is set accordingly, that is, the closer the predicted future track point of the unmanned platform is to the curve of the function, the higher the reward score is, such as Figure 2 As shown, the fitting method includes the following steps:

[0055] S201 determines the interpolation points: In this embodiment, the interpolation points are the existing track point observation data. The area between every two adjacent track points is used as the interpolation area. The points inserted later in the interpolation area are used as the track supplement, so as to perform track fitting and obtain the historical track.

[0056] S202 constructs interpolation polynomial: divide the interpolation interval into multiple subintervals [x i ,x i+1 ], and use any two adjacent interpolation points (x i ,y i )、(xi+1 ,y i+1 ) construct the corresponding linear polynomial;

[0057] The constructed linear polynomial is:

[0058]

[0059] Among them, x is the interpolation point, x i is the i-th interpolation point, x i+1 is the i+1th interpolation point, y i is x i The corresponding function value, y i+1 is x i+1 The function value corresponding to the interpolation point; the values ​​of the interpolation function on both sides of the division point of adjacent subintervals are equal, that is, P i (x i+1 )=P i+1 (x i+1 ).

[0060] S203 uses the linear polynomial (1) to calculate the function value corresponding to the non-interpolation point in the corresponding subinterval to obtain the interpolation function;

[0061] S204 compares the actual data to verify the accuracy of the interpolation function; specifically, by comparing with the actual track point data, the accuracy and effect of the interpolation function are verified, and a piecewise function constructed by the linear interpolation method (i.e., the historical track fitting function) is obtained, which can be used to approximate the original function.

[0062] S3 builds scene parameters: Based on the static and dynamic obstacle information in the application scenario, a corresponding repulsion field is constructed. Specifically, the static and dynamic obstacle information in the scene is input, and a repulsion field is formed for each obstacle. Each obstacle and the corresponding repulsion field information are used as one of the important indicators for future trajectory prediction.

[0063] Keeping the predicted track away from obstacles is also one of the goals of the track prediction algorithm. Therefore, a penalty function is set. That is, if the predicted future track point of the unmanned platform falls within the obstacle threat radius, a certain reward point will be deducted. When predicting the next track point of the unmanned platform, the influence of the repulsive field is first considered, and the track prediction point with smaller repulsive force is given priority.

[0064] In step S3, the repulsive field is constructed using the first formula:

[0065]

[0066] Among them, F is the magnitude of the repulsive force, gain rp is the repulsive force constant, dis pois the actual distance between the unmanned platform and the center of mass of a static obstacle or the center of mass of a dynamic obstacle. imp Is the range of influence of static obstacles or dynamic obstacles, area fb It is the prohibited range of static obstacles or dynamic obstacles.

[0067] S4 algorithm parameter initialization and model training: Initialize the algorithm and set the reward and punishment function based on the historical track, static obstacle information, dynamic obstacle information and repulsion field information, and then use the reward and punishment function to train the network model; specifically, the initialization algorithm includes initializing the environment, setting the initial state, initializing the unmanned platform parameters, initializing the playback memory, etc. At the same time, the reward and punishment function is set based on the acquired historical track fitting function, dynamic obstacle information and static obstacle information.

[0068] 1) Initialize the environment

[0069] The environment defines the state of the system and its possible actions, as well as the rules for how these actions change the state. In this embodiment, the environment is represented by (time range, longitude range, latitude range, altitude range), that is, the environment env = [[0, -180, -90, -1000], [1000, 180, 90, 1000]], and the action space act = [[-1], [1]].

[0070] 2) Set the initial state

[0071] Set the initial state of the unmanned platform, including timestamp, longitude, latitude, altitude, heading, speed, and pitch angle. Heading is 0 degrees from true north and increases clockwise. The pitch angle is the angle between the platform's nose and the horizontal plane of the ground coordinate system. Platform speed is uniformly measured in m / s. In this example, the initial state fs = [0, 120, 30, 0, 120, 10, 0].

[0072] 3) Initialize unmanned platform parameters

[0073] Unmanned platform parameters include the maximum speed, cruising range, maximum steering angle, etc.

[0074] 4) Set up reward and punishment functions

[0075] The closer the unmanned platform's track points fit the fitted track curve, the higher the score. The distance between the unmanned platform and the track curve ranges from [0, +∞]. Calculate the repulsion F of the obstacle on the unmanned platform. The greater the repulsion F, the less reward. During the training and learning process of the intelligent agent, the size of the repulsion needs to be considered to determine the location of the next path point. At the same time, if the unmanned platform's track point falls within the threat range of the obstacle, it will be penalized. The distance between the track point and the center of mass of the obstacle is dis po The value minus the distance between the intersection of the track point and the obstacle boundary and the obstacle center is the actual distance between the unmanned platform and the obstacle boundary. pa , will dis pa As one of the parameters of the reward and punishment function, namely dis pa When it is a negative number, the larger the absolute value, the more severe the penalty. pa When it is a positive number, the larger the absolute value, the greater the reward.

[0076] The reward and punishment function is set as:

[0077]

[0078] Among them, F is the magnitude of the repulsive force, dis l is the distance between the current track point of the unmanned platform and the fitted route, dis pa It is the actual distance between the unmanned platform and the obstacle boundary.

[0079] 5) Model training

[0080] The network model is trained using the experience replay mechanism until the prediction accuracy of the trained network model meets the preset requirements. The training is stopped. The strategy network selected in this embodiment is the Q network.

[0081] like Figure 3 As shown, the method for training the network model using the experience replay algorithm includes the following steps:

[0082] S401 In each time step, the unmanned platform selects and executes an action based on the greedy strategy according to the current value table; then the current environment gives rewards or penalties based on the reward and punishment function, as well as information on whether the process is over; the unmanned platform observes the current environment and stores the current state, executed action, next state, and reward or penalty obtained as experience data in the experience replay buffer; specifically, the experience replay buffer is a storage structure (such as an array or queue) created to store experience tuples, each of which contains the state s at time t. t 、Action a t and reward r t , and the new state s at time t+1t+1 , store each experience tuple as a four-tuple (s t ,a t ,r t ,s t+1 ) and adds it to the experience replay buffer.

[0083] S402 When there is enough data in the experience replay buffer, a small batch of samples is randomly sampled for training. These samples are used to calculate the loss function and update the parameters of the Q network. The purpose of random sampling is to break the correlation between the data and make the training process more stable.

[0084] Specifically, a batch of experience data is randomly extracted from the experience replay buffer to update the Q network parameters of the unmanned platform; the Q network parameters are updated using the mean square error loss and gradient descent method, and the Q network parameters are copied to the Q′ network at regular step lengths. The update formula is:

[0085]

[0086] Where L(θ) is the loss function; maxQ'(s',a') is the maximum Q value of all possible actions in the next state (s'); s' is the state reached after performing action aˊ; θ is the Q network parameter, s is the state reached after performing action a; r is the immediate reward, γ is the discount factor used to balance the immediate reward and future rewards; E is the expectation.

[0087] S403 repeats steps S401 to S402, iteratively updates the target network, and stops training when the prediction accuracy of the trained network model meets the preset requirements.

[0088] Specifically, the above process is repeated, allowing the unmanned platform to continuously learn and explore the environment. After a certain period of time, actions are selected based on the strategy, and the data obtained from the exploration is used to update the value function or strategy. The training process ends when the predicted trajectory meets the accuracy requirements.

[0089] S5 track prediction

[0090] The model trained in step S4 is used to predict the trajectory of the unmanned platform in the future. That is, based on the regular experience learned by the model, the trajectory of the unmanned platform in the future, such as the next 60 seconds, is predicted.

[0091] This embodiment introduces an experience replay mechanism and an improved potential field method, which improves learning efficiency and stability, while reducing dependence on the model, lowering the difficulty and cost of modeling; it also uniformly integrates the motion characteristics (such as heading and pitch angle) of various unmanned platforms such as drones and unmanned ships, so that the predicted unmanned platform track can truly cover the entire environmental space, has strong adaptability, can handle higher-dimensional state and action spaces, and is suitable for more complex application scenarios.

[0092] Based on the methods in the above embodiments, an embodiment of the present application provides an electronic device, which may include: a processor 110, a communication interface 120, a memory 130, and a communication bus 140, wherein the processor 110, the communication interface 120, and the memory 130 communicate with each other via the communication bus 140. The processor 110 may call logic instructions in the memory 130 to execute the methods in the above embodiments.

[0093] In addition, the logic instructions in the above-mentioned memory 130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0094] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0095] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0096] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0097] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0098] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0099] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0100] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for unmanned platform track prediction based on reinforcement learning and improved potential field method, characterized in that: include: S1 obtains the track parameters of the unmanned platform in a continuous time period as sample data; S2 fits the historical track of the unmanned platform using a function fitting method based on the sample data; S3 constructs the corresponding repulsive field based on the static and dynamic obstacle information in the application scenario using the following formula: in, F is the magnitude of the repulsive force, gain rp is the repulsive force constant, dis po is the actual distance between the unmanned platform and the center of mass of a static obstacle or the center of mass of a dynamic obstacle. area imp is the influence range of static obstacles or dynamic obstacles, area fb It is the prohibited area for static obstacles or dynamic obstacles; S4 initializes experience replay algorithm parameters, sets a reward and punishment function based on the historical track, static obstacle information, dynamic obstacle information, and the repulsive field, and trains the network model using the reward and punishment function; S5 uses the network model trained in step S4 to predict the trajectory of the unmanned platform in the future.

2. The unmanned platform track prediction method based on reinforcement learning and improved potential field method according to claim 1, characterized in that: In step S2, the method of fitting the historical track using a function fitting method includes the following steps: S201: taking the area between every two adjacent track points as an interpolation area, and interpolating in the interpolation area to supplement the track; S202: dividing the interpolation area after the supplemented track into a plurality of subintervals, and constructing corresponding linear polynomials using any two adjacent interpolation points in the subintervals; wherein the values ​​of the interpolation functions on the left and right sides of the division points of adjacent subintervals are equal; S203 uses the linear polynomial to calculate the function value corresponding to the non-interpolation point in the corresponding subinterval to obtain a piecewise function; S204 compares the actual track point data to verify the accuracy of the piecewise function to obtain the historical track.

3. The unmanned platform track prediction method based on reinforcement learning and improved potential field method according to claim 2, characterized in that: The linear polynomial is: in, x is the interpolation point, x i is the i-th interpolation point, x i+1 is the i+1th interpolation point, y i yes x i The corresponding function value is y i+1 yes x i+1 The function value corresponding to the interpolation point.

4. The unmanned platform track prediction method based on reinforcement learning and improved potential field method according to claim 1, characterized in that: In step S1, the track parameters include longitude, latitude, heading, speed and pitch angle.

5. The unmanned platform track prediction method based on reinforcement learning and improved potential field method according to claim 1, characterized in that: The reward and punishment function is: in, F is the magnitude of the repulsive force, dis l is the distance between the current track point of the unmanned platform and the fitted route, dis pa It is the actual distance between the unmanned platform and the obstacle boundary.

6. The unmanned platform trajectory prediction method based on reinforcement learning and improved potential field method according to claim 1, wherein the method of training the network model using the experience replay algorithm comprises the following steps: S401: In each time step, the unmanned platform selects and executes an action based on the greedy strategy according to the current value table. The current environment gives rewards or penalties based on the reward and penalty function. The unmanned platform observes the current environment and stores the current state, executed action, next state, and reward or penalty obtained as experience data in the experience replay buffer. S402 randomly extracts a batch of experience data from the experience replay buffer to update the Q network of the unmanned platform; uses the mean square error loss and gradient descent method to update the Q network parameters, and copies the Q network parameters to the Q′ network at regular step lengths, where the update formula is: in, L ( θ ) is the loss function; max Q ' (s′,a′) Is the next state （s'） The maximum Q value of all possible actions; s′ Is to perform an action aʹ The state achieved after θ for Q Network parameters, s Is to perform an action a The state achieved after r It’s an instant reward. γ is a discount factor used to balance immediate rewards and future rewards; E It is expectation; S403 repeats steps S401 to S402, iteratively updates the target network, and stops training when the prediction accuracy of the trained network model meets the preset requirements.

7. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed on a processor, the processor is caused to execute the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that When the computer program product is run on a processor, the processor is enabled to perform the method according to any one of claims 1 to 6.

Images