Target tracking and data interconnection method based on reinforcement learning
By designing reinforcement learning networks and Bayesian networks to optimize point-track correlation, the problem of precise interconnection of traditional data interconnection algorithms in unknown environments is solved, and multi-objective tracking and data interconnection in complex environments is realized, and accurate track output is output.
Patent Information
- Application Number
- CN202510379931.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-08-01
AI Technical Summary
Traditional data interconnection algorithms cannot work effectively in the unknown target system model, and are affected by environmental clutter, sensor measurement errors and target strong maneuvering, resulting in inaccurate interconnection results.
Design reinforcement learning networks, including environmental state, action space, reward function and policy network, through single-objective learning and multi-objective testing processes, use Bayesian networks and long-term memory networks to optimize point-track correlation and track generation, and output complete and accurate tracks.
Achieve accurate interconnection of multiple goals in complex environments, reduce the influence of external factors, and the output track is similar to the real results, with a wide range of applications and good practical results.
Smart Images
Figure CN120405647A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for target tracking and data association based on reinforcement learning, which is particularly applicable to the problems of target tracking and data association in the field of information fusion, and belongs to the technical fields of reinforcement learning and target tracking. Background Art
[0002] Data association is a key technology in the field of radar data processing. Its core lies in establishing the relationship between radar measurement data at adjacent sampling times, and determining the true tracks and trails based on this. Usually, traditional data association methods center around determining the predicted values of tracks, screening eligible trails according to a certain specific criterion, and using the trails for filtering. These methods are generally related to filtering algorithms in target tracking, such as probabilistic data association filtering algorithm, joint probabilistic data association filtering algorithm, multiple hypothesis tracking filtering algorithm, interacting multiple model filtering algorithm, probability hypothesis density filtering algorithm, etc. In some specific environments, through some special means, these algorithms can handle some target data association problems. However, they have not completely gotten rid of the limitations of the traditional filtering framework and need to implement data association based on the system model. Obviously, the system model in the real environment is unpredictable, which is also one of the reasons why traditional data association algorithms have poor practicability.
[0003] In recent years, with the rise of the field of artificial intelligence, using artificial intelligence means to analyze data has become the mainstream of the times. Reinforcement learning is an important branch in the field of machine learning, which emphasizes how to act based on the environment to obtain the maximum expected benefit. After decades of development, there have been many achievements in reinforcement learning technology. For example, Q-learning, dynamic programming, Policy Gradients, Deep-Q-Network, etc. have also been applied to fields such as games, film and television, and transportation. Essentially, reinforcement learning is a process of allowing a machine to self-learn in an unknown environment under established rules. The behavior of the machine conforms to the real environment, and rewards or punishments are feedback by the established rules, with reaching the "end point" as the ending mark. Simply put, the process of target data association is to find the true trails generated by the target track, and all the trails are sorted along the time axis to form a complete track. This process is both similar to finding the optimal path and can roughly be regarded as a snake game. The target environment is the environment that the machine needs to adapt to, and the game rules are determined by the target motion state. Therefore, from a theoretical perspective, reinforcement learning can solve the problems of target tracking and data association. However, how to rationally apply reinforcement learning technology based on the traditional data association idea to achieve precise association of multiple targets in a complex environment with scarce prior information is an urgent problem to be solved. Summary of the Invention
[0004] A target tracking and data association method based on reinforcement learning according to the present invention aims to overcome the defect that traditional data association algorithms cannot work effectively when the target system model is unknown, and eliminate the influence of factors such as environmental clutter, sensor measurement error, and strong target maneuver on the association result as much as possible.
[0005] A target tracking and data association method based on reinforcement learning according to the present invention is characterized in that it includes the following steps:
[0006] Step 1: Design the basic elements of the reinforcement learning network, namely the environmental state, action space, reward function, and policy network;
[0007] Step 2: Enter the training session of the single-target learning process;
[0008] Step 3: After the training is completed, enter the testing session of the single-target learning process;
[0009] Step 4: If the test result is a complete track and is similar to the true result, the learning ends and step 5 is entered; otherwise, return to step 2 and re-learn;
[0010] Step 5: After the single-target learning process ends, enter the multi-target testing process;
[0011] Preferably, the specific steps of step 1 are as follows:
[0012] Step 1.1: Define the environmental state as the tracks associated with five consecutive sampling moments;
[0013] Step 1.2: The action space consists of the tracks obtained at each sampling moment, and each action corresponds to the track associated with the current sampling moment;
[0014] Step 1.3: Define the reward function, that is, in the state at the current moment, first predict the order of the least squares method based on the Bayesian network, then use the least squares method to predict the position of the target at the next moment, and finally calculate the reward value of the selected action through the Bayesian recursion function;
[0015] Step 1.4: The policy network uses a long short-term memory network. The input end of the network is the state at the current moment and a single track at the next moment, and the output end of the network is the association probability between the track and the state.
[0016] Preferably, the specific steps of step 2 are as follows:
[0017] Step 2.1: Detect the environmental state from the acquired measurement data and determine the environmental state value;
[0018] Step 2.2: Randomly select a track from the track data obtained at the next sampling moment as the action of the environmental state at the current sampling moment;
[0019] Step 2.3: Input the environmental state and the selected action at the current moment into the policy network to obtain the correlation probability between the state and the tracklet.
[0020] Step 2.4: Input the environmental state and the selected action at the current moment into the reward function to calculate the true score of the selected action in the current state.
[0021] Step 2.5: Feed back the environmental state, action, reward value, and correlation probability to the policy network for learning, and train until the complete track is continuously output as the end point until the training is completed.
[0022] Preferably, step 2.1 specifically includes the following sub-steps:
[0023] Step 2.1.1: Traverse all tracklets at five consecutive sampling moments in an exhaustive manner, and find all possible existing tracks according to the requirement that the tracklets at adjacent sampling moments must meet the speed threshold.
[0024] Step 2.1.2: According to the cosine theorem formula, the motion trend factor of the track at three consecutive sampling moments can be calculated, and in this way, the motion trend factors at five consecutive sampling moments are calculated to form the motion trend set of all tracks.
[0025] Step 2.1.3: Solve the variance value of the motion trend set of all tracks, and find the track with the smallest variance value as the environmental state value.
[0026] Preferably, step 2.4 specifically includes the following sub-steps:
[0027] Step 2.4.1: Input the environmental state at the current moment into the three-class Bayesian network (the number of classifications represents the order of the least squares method) to obtain the probabilities of the three classifications, and select the largest classification number as the order of the least squares method.
[0028] Step 2.4.2: Fit the state data using the least squares method of the selected order to predict the target position at the next sampling moment.
[0029] Step 2.4.3: Input the predicted position and the selected action into the Bayesian recursive function to calculate the reward value of the selected action and obtain the true score.
[0030] Preferably, the specific steps of step 3 are as follows:
[0031] Step 3.1: Facing the test data set, execute step 2.1.
[0032] Step 3.2: Input the environmental state at the current moment and the tracklet data obtained at the next sampling moment into the policy network to obtain the correlation probability between the state and the tracklet.
[0033] Step 3.3: According to the magnitude of the association probability value, select the track with the largest probability value for the environmental state as the corresponding action;
[0034] Step 3.4: Repeat the process until the test ends;
[0035] Preferably, the specific steps of step 3.2 are as follows:
[0036] Step 3.2.1: Match the environmental state with the track data to generate a combination of one state and one track;
[0037] Step 3.2.2: Input all combinations into the policy network to obtain the association probability of each combination;
[0038] Preferably, the specific steps of step 5 are as follows:
[0039] Step 5.1: Detect the environmental state from the acquired measurement data to determine the environmental state values of all targets;
[0040] Step 5.2: Based on all the environmental states at the current moment and all the track data obtained at the next sampling moment, execute step 3.2;
[0041] Step 5.3: Based on the obtained association probability, select a suitable track as the action for each environmental state;
[0042] Step 5.4: Input the environmental state at the current moment and the selected action into the reward function, execute step 2.4, and calculate the true score of the action selected for each state;
[0043] Step 5.5: Repeat the process until the test ends.
[0044] Preferably, the specific steps of step 5.1 are as follows:
[0045] Step 5.1.1: Adopt an exhaustive method to traverse all the tracks at five consecutive sampling moments. According to the requirement that the tracks at adjacent sampling moments must meet the speed threshold, find all possible existing tracks;
[0046] Step 5.1.2: Classify the tracks according to the principle of "one-to-one correspondence between the target and the track" to ensure that there is no track intersection between different groups of tracks;
[0047] Step 5.1.3: According to the cosine theorem formula, the motion trend factor of the track at three consecutive sampling moments can be calculated, and in this way, the motion trend factors at five consecutive sampling moments are calculated to form the motion trend set of all tracks;
[0048] Step 5.1.4: Solve the variance values of all sets of track movement trends, find the track with the smallest variance value in each type of track, and use it as the environmental state value, where the category corresponds to the number of targets;
[0049] Preferably, the said Step 5.3 specifically includes the following sub-steps:
[0050] Step 5.3.1: According to the magnitude of the association probability value, select the plot with the largest probability value for each environmental state as the corresponding action;
[0051] Step 5.3.2: If there is a "competition" situation where multiple environmental states select the same plot, then allocate the plot according to the rules of "there is only one plot that may originate from the target for the state" and "the interconnection probability between the plot and the state is the largest" until all states are associated with plots;
[0052] Step 5.3.3: If the number of plots that can be selected for an environmental state is not unique, then select the plot according to the rules of "each plot has only a unique source target" and "for a given target, at most one plot has it as the source", and select the plot with the largest interconnection probability for each state.
[0053] A method for target tracking and data interconnection based on reinforcement learning proposed by the present invention. First, design the basic elements of the reinforcement learning network, namely environmental state, action space, reward function, and policy network. Then, start the training session and test session of the single-target learning process. In these two sessions, loop in the ways of "determine the environmental state - randomly select an action - output the association probability - calculate the action reward - feedback network learning" and "determine the environmental state - output the association probability - select the action with the largest probability value" respectively, with continuously outputting a complete track as the end point of the training session and the test result being a complete track and similar to the true result as the sign of the end of learning. Finally, after learning with a large number of training data sets, enter the multi-target test process. In this process, loop in the way of "determine all environmental states - output the association probability - match a suitable action for each target with the criterion that the larger the probability value, the better - calculate all action rewards", and output the final test result. The present invention can learn through a large number of training sample data and, based on this, conduct tests to obtain the final test result, having the advantages of a wide application range, little influence from external factors, and good practical effects. The proposed method for target tracking and data interconnection can be directly applied to corresponding practical problems and has high practical application value. Description of the Drawings
[0054] Figure 1 is the flowchart of the single-target learning process of a method for target tracking and data interconnection based on reinforcement learning of the present invention;
[0055] Figure 2 It is the multi-objective test process flow chart of a method for target tracking and data interconnection based on reinforcement learning in the present invention. Specific implementation manners
[0056] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.
[0057] Embodiment 1
[0058] A method for target tracking and data interconnection based on reinforcement learning in this embodiment refers to the attached Figure 1-2 , and includes the following steps:
[0059] Step 1: Design the basic elements of the reinforcement learning network, namely the environmental state, action space, reward function, and policy network;
[0060] Step 1.1: Define the environmental state as the traces associated with five consecutive sampling moments;
[0061] Step 1.2: The action space consists of the traces obtained at each sampling moment, and each action corresponds to the trace associated with the current sampling moment;
[0062] Step 1.3: Define the reward function. That is, in the state at the current moment, first predict the order of the least squares method based on the Bayesian network, then use the least squares method to predict the position of the target at the next moment, and finally calculate the reward value of the selected action through the Bayesian recursion function;
[0063] Step 1.4: The policy network uses a long short-term memory network. The input end of the network is the state at the current moment and a single trace at the next moment, and the output end of the network is the association probability between the trace and the state;
[0064] Step 2: Enter the training session of the single-target learning process;
[0065] Step 2.1: Detect the environmental state from the obtained measurement data and determine the environmental state value;
[0066] Step 2.1.1: Adopt an exhaustive method to traverse all the traces of five consecutive sampling moments, and find all possible tracks according to the requirement that the traces at adjacent sampling moments must meet the speed threshold;
[0067] Step 2.1.2: According to the cosine theorem formula, the motion trend factor of the track for three consecutive sampling moments can be calculated, and in this way, the motion trend factors for five consecutive sampling moments are calculated to form the motion trend set of all tracks;
[0068] Step 2.1.3: Solve the variance value of all track motion trend sets, and find the track with the smallest variance value as the environmental state value;
[0069] Step 2.2: Randomly select a plot from the plot data obtained at the next sampling moment as the action of the environmental state at the current sampling moment;
[0070] Step 2.3: Input the environmental state and the selected action at the current moment into the policy network to obtain the correlation probability between the state and the plot;
[0071] Step 2.4: Input the environmental state and the selected action at the current moment into the reward function to calculate the true score of the selected action in the current state;
[0072] Step 2.4.1: Input the environmental state at the current moment into the three-class Bayesian network (the number of classifications represents the order of the least squares method) to obtain the probabilities of the three classifications, and select the largest classification number as the order of the least squares method;
[0073] Step 2.4.2: Use the least squares method of the selected order to fit the state data and predict the target position at the next sampling moment;
[0074] Step 2.4.3: Input the predicted position and the selected action into the Bayesian recursive function to calculate the reward value of the selected action and obtain the true score;
[0075] Step 2.5: Feed back the environmental state, action, reward value, and correlation probability to the policy network for learning, and train until the end with the continuous output of the complete track as the end point;
[0076] Step 3: After the training is completed, enter the test session of the single-target learning process;
[0077] Step 3.1: In the face of the test data set, execute Step 2.1;
[0078] Step 3.2: Input the environmental state at the current moment and the plot data obtained at the next sampling moment into the policy network to obtain the correlation probability between the state and the plot;
[0079] Step 3.2.1: Match the environmental state and the plot data to generate a combination of a state and a plot;
[0080] Step 3.2.2: Input all combinations into the policy network to obtain the correlation probability of each combination;
[0081] Step 3.3: According to the magnitude of the correlation probability value, select the plot with the largest probability value for the environmental state as the corresponding action;
[0082] Step 3.4: Repeat in a loop until the test is completed;
[0083] Step 4: If the test result is a complete track and is similar to the true result, the learning ends and proceeds to Step 5; otherwise, return to Step 2 and re-learn;
[0084] Step 5: After the single-target learning process ends, enter the multi-target testing process;
[0085] Step 5.1: Detect the environmental state from the acquired measurement data to determine the environmental state values of all targets;
[0086] Step 5.1.1: Adopt an exhaustive method to traverse all the traces at five consecutive sampling moments. According to the requirement that the traces at adjacent sampling moments must meet the speed threshold, find all possible existing tracks;
[0087] Step 5.1.2: Classify the tracks according to the principle of "one-to-one correspondence between the target and the trace" to ensure that there is no trace intersection between different groups of tracks;
[0088] Step 5.1.3: According to the cosine theorem formula, the motion trend factor of the track at three consecutive sampling moments can be calculated, and in this way, the motion trend factors at five consecutive sampling moments are calculated to form the motion trend set of all tracks;
[0089] Step 5.1.4: Solve the variance value of the motion trend set of all tracks, and find the track with the smallest variance value in each type of track as the environmental state value, and the number of targets corresponding to the category;
[0090] Step 5.2: Based on all the environmental states at the current moment and all the trace data obtained at the next sampling moment, execute Step 3.2;
[0091] Step 5.3: Based on the obtained association probability, select a suitable trace as the action for each environmental state;
[0092] Step 5.3.1: According to the magnitude of the association probability value, select the trace with the largest probability value as the corresponding action for each environmental state;
[0093] Step 5.3.2: If there is a "competition" situation where multiple environmental states select the same trace, then allocate the trace according to the rules of "there is only one trace that may originate from the target for a state" and "the interconnection probability between the trace and the state is the largest" until all states are associated with traces;
[0094] Step 5.3.3: If the number of traces that can be selected for an environmental state is not unique, then select the trace according to the rules of "each trace has only a unique source target" and "for a given target, at most one trace has it as the source", and select the trace with the largest interconnection probability for each state;
[0095] Step 5.4: Input the environmental state and the selected action at the current moment into the reward function, execute Step 2.4, and calculate the true score of the selected action for each state.
[0096] Step 5.5: Repeat the process until the test ends.
[0097] Embodiment 2
[0098] To better illustrate the present invention, this embodiment takes the measurement data of a certain type of radar as a specific example to elaborate on the steps of the present invention in detail:
[0099] Step 11: Design the basic elements of the reinforcement learning network, namely the environmental state, action space, reward function, and policy network;
[0100] Step 11.1: If Z t is the measurement set at time t, then is the state composed of the traces originating from the nth target in {Z t-4 , Z t-3 , Z t-2 , Z t-1 , Z t} at time t.
[0101] Step 11.2: The action space is composed of the traces obtained at each sampling moment, and each action corresponds to the trace associated with the current sampling moment.
[0102] Step 11.3: Define the reward function. That is, in the state at the current moment, first predict the order of the least squares method based on the Bayesian network, then use the least squares method to predict the position of the target at the next moment, and finally calculate the reward value of the selected action through the Bayesian recursive function, that is
[0103]
[0104]
[0105] where represents the ith trace selected from the measurement set at time t; K t represents the clutter intensity at time t, that is num t is the number of traces in the measurement set, TS is the area of the radar detection area; R is the measurement covariance matrix, which is determined by the radar measurement error; P_D is the detection probability.
[0106] Step 11.4: The policy network uses a long short-term memory network. The input end of the network is the state at the current moment and a single trace at the next moment, and the output end of the network is the association probability between the trace and the state, and the value range is [0, 1];
[0107] Step 12: Enter the training session of the single-target learning process;
[0108] Step 12.1: Detect the environmental state from the acquired measurement data and determine the environmental state value;
[0109] Step 12.1.1: According to the sampling interval T_sample of the sensor, the measurement data z, the maximum speed v_max and the minimum speed v_min of the target movement, it is required that the tracks at adjacent times must meet the speed threshold, that is
[0110]
[0111] Step 12.1.2: According to the cosine theorem formula, the motion trend factor of the track at three consecutive sampling times can be calculated, that is
[0112]
[0113] The motion trend at five consecutive times can be calculated according to the above formula Furthermore, the motion trends of all tracks are obtained;
[0114] Step 12.1.3: Solve the variance value Variance = var(f) of the set of motion trends of all tracks, and find the track with the smallest variance value as the environmental state value;
[0115] Step 12.2: Randomly select a track from the track data obtained at the next sampling time as the action of the environmental state at the current sampling time;[[ID=2x]]
[0116] Step 12.3: Input the environmental state and the selected action at the current time into the policy network to obtain the correlation probability between the state and the track;
[0117] Step 12.4: Input the environmental state and the selected action at the current time into the reward function to calculate the true score of the selected action in the current state
[0118] Step 12.4.1: Input the environmental state at the current time into the three-class Bayesian network (the number of classifications represents the order of the least squares method) to obtain the probabilities of the three classifications, and select the largest classification number as the order g of the least squares method;
[0119] Step 12.4.2: Use the least squares method of the selected order to fit the state data and predict the target position at the next sampling time;
[0120] Step 12.4.3: Input the predicted position and the selected action into the Bayesian recursive function to calculate the reward value of the selected action and obtain the true score
[0121] Step 12.5: Feed the environmental state, action, reward value, and association probability back to the policy network for learning, and conduct training until the complete track is continuously output as the end point until the training is completed;
[0122] Step 13: After the training is completed, enter the testing phase of the single-target learning process;
[0123] Step 13.1: Facing the test data set, execute Step 2.1;
[0124] Step 13.2: Input the environmental state at the current moment and the point track data obtained at the next sampling moment into the policy network together to obtain the association probability between the state and the point track;
[0125] Step 13.2.1: Match the environmental state with the point track data to generate a combination of one state and one point track;
[0126] Step 13.2.2: Input all combinations into the policy network to obtain the association probability of each combination;
[0127] Step 13.3: According to the magnitude of the association probability value, select the point track with the largest probability value for the environmental state as the corresponding action;
[0128] Step 13.4: Repeat in a loop until the test is completed;
[0129] Step 14: If the test result is a complete track and is similar to the true result, the learning is completed and enter Step 5; otherwise, return to Step 2 and re-learn;
[0130] Step 15: After the single-target learning process is completed, enter the multi-target testing process;
[0131] Step 15.1: Detect the environmental state from the obtained measurement data to determine the environmental state values of all targets;
[0132] Step 15.1.1: According to the sampling interval T_sample of the sensor, the measurement data z, the maximum speed v_max and the minimum speed v_min of the target movement, it is required that the point tracks at adjacent moments must meet the speed threshold, that is
[0133]
[0134] Step 1..2: Classify the tracks according to the principle of "one-to-one correspondence between the target and the point track" to ensure that there is no point track intersection between tracks in different groups;
[0135] Step 15.1.3: According to the cosine theorem formula, the motion trend factor of the track at three consecutive sampling moments can be calculated, that is
[0136]
[0137] According to the above formula, the motion trends at five consecutive moments can be calculated. Furthermore, the motion trends of all tracks can be obtained.
[0138] Step 15.1.4: Solve the variance value Variance = var(f) of the set of motion trends of all tracks, and find the track with the smallest variance value in each type of track as the environmental state value, and the category corresponds to the number of targets.
[0139] Step 15.2: Based on all the environmental states at the current moment and all the point track data obtained at the next sampling moment, execute Step 3.2.
[0140] Step 15.3: Based on the obtained association probabilities, select appropriate point tracks as actions for each environmental state.
[0141] Step 15.3.1: According to the magnitudes of the association probability values, select the point track with the largest probability value as the corresponding action for each environmental state.
[0142] Step 15.3.2: If there is a "competition" situation where multiple environmental states select the same point track, then allocate the point track according to the rules of "there is only one point track that may originate from the target for the state" and "the interconnection probability between the point track and the state is the largest" until all states are associated with point tracks.
[0143] Step 15.3.3: If the number of point tracks that can be selected for an environmental state is not unique, then select the point track according to the rules of "each point track has only a unique source target" and "for a given target, there is at most one point track with it as the source", and select the point track with the largest interconnection probability for each state.
[0144] Step 15.4: Input the environmental state at the current moment and the selected actions into the reward function, execute Step 2.4, and calculate the true scores of the actions selected for each state.
[0145] Step 15.5: Repeat in a loop until the test ends.
[0146] As described above, it is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its inventive concept, makes equivalent substitutions or changes, and all should be covered within the protection scope of the present invention.
Claims
1. A target tracking and data interconnection method based on reinforcement learning, characterized in that It includes the following steps: Step 1: Design the basic elements of the reinforcement learning network, namely the environmental state, action space, reward function, and policy network; Step 2: Enter the training session of the single-object learning process; Step 3: After the training ends, enter the testing session of the single-object learning process; Step 4: If the test result is a complete track and is similar to the true result, the learning ends and enter Step 5; otherwise, return to Step 2 and re-learn; Step 5: After the single-object learning process ends, enter the multi-object testing process.
2. The method for target tracking and data interconnection based on reinforcement learning according to claim 1, characterized in that The specific steps of Step 1 are as follows: Step 1.1: Define the environmental state as the point tracks associated with 5 consecutive sampling moments; Step 1.2: The action space consists of the point tracks obtained at each sampling moment, and each action corresponds to the point track associated with the current sampling moment; Step 1.3: Define the reward function, that is, in the state at the current moment, first predict the order of the least squares method based on the Bayesian network, then use the least squares method to predict the position of the target at the next moment, and finally calculate the reward value of the selected action through the Bayesian recursive function; Step 1.4: The policy network uses a long short-term memory network. The input end of the network is the state at the current moment and a single point track at the next moment, and the output end of the network is the association probability between the point track and the state.
3. A method for target tracking and data interconnection based on reinforcement learning according to claim 1, characterized in that The specific steps of Step 2 are as follows: Step 2.1: Detect the environmental state from the acquired measurement data and determine the environmental state value; Step 2.2: Randomly select a point track from the point track data obtained at the next sampling moment as the action of the environmental state at the current sampling moment; Step 2.3: Input the environmental state and the selected action at the current moment into the policy network to obtain the association probability between the state and the point track; Step 2.4: Input the environmental state and the selected action at the current moment into the reward function to calculate the true score of the selected action in the current state; Step 2.5: Feed back the environmental state, action, reward value, and association probability to the policy network for learning, and train until the training ends with the continuous output of a complete track as the end point.
4. A method for target tracking and data interconnection based on reinforcement learning according to claim 3, characterized in that The specific steps of Step 2.1 include the following sub-steps: Step 2.1.1: Use the exhaustive method to traverse all the point tracks of 5 consecutive sampling moments, and find all possible existing tracks according to the requirement that the point tracks at adjacent sampling moments must meet the speed threshold; Step 2.1.2: According to the cosine theorem formula, the motion trend factor of the track for 3 consecutive sampling moments can be calculated, and in this way, the motion trend factors for 5 consecutive sampling moments are calculated to form the motion trend set of all tracks; Step 2.1.3: Solve the variance value of all track motion trend sets and find the track with the smallest variance value as the environmental state value.
5. A method for target tracking and data interconnection based on reinforcement learning according to claim 3, characterized in that The specific steps of Step 2.4 include the following sub-steps: Step 2.4.1: Input the environmental state at the current moment into the three-class Bayesian network (the number of classifications represents the order of the least squares method) to obtain the probabilities of the three classifications, and select the largest classification number as the order of the least squares method; Step 2.4.2: Use the least squares method with the selected order to fit the state data and predict the target position at the next sampling moment; Step 2.4.3: Input the predicted position and the selected action into the Bayesian recursive function to calculate the reward value of the selected action and obtain the true score.
6. A method for target tracking and data interconnection based on reinforcement learning according to claim 1, characterized in that The specific steps of step 3 are as follows: Step 3.1: Facing the test data set, execute step 2.1; Step 3.2: Input the environmental state at the current moment and the track data obtained at the next sampling moment into the policy network to obtain the correlation probability between the state and the track; Step 3.3: According to the magnitude of the correlation probability value, select the track with the largest probability value for the environmental state as the corresponding action; Step 3.4: Repeat the process until the test ends.
7. A method for target tracking and data interconnection based on reinforcement learning according to claim 1, characterized in that Step 3.2 specifically includes the following sub-steps: Step 3.2.1: Match the environmental state with the track data to generate a combination of one state and one track; Step 3.2.2: Input all combinations into the policy network to obtain the correlation probability of each combination.
8. A method for target tracking and data interconnection based on reinforcement learning according to claim 1, characterized in that The specific steps of step 5 are as follows: Step 5.1: Detect the environmental state from the obtained measurement data to determine the environmental state values of all targets; Step 5.2: Based on all the environmental states at the current moment and all the track data obtained at the next sampling moment, execute step 3.2; Step 5.3: Based on the obtained correlation probability, select a suitable track as the action for each environmental state; Step 5.4: Input the environmental state at the current moment and the selected action into the reward function, execute step 2.4, and calculate the true score of the selected action for each state; Step 5.5: Repeat the process until the test ends.
9. A method for target tracking and data interconnection based on reinforcement learning according to claim 8, characterized in that, Step 5.1 specifically includes the following sub-steps: Step 5.1.1: Adopt an exhaustive method to traverse all the tracks at five consecutive sampling moments. According to the requirement that the tracks at adjacent sampling moments must meet the speed threshold, find all possible tracks; Step 5.1.2: Classify the tracks according to the principle of "one-to-one correspondence between the target and the track" to ensure that there is no track intersection between different groups of tracks; Step 5.1.3: According to the cosine theorem formula, the motion trend factor of the track at three consecutive sampling moments can be calculated, and in this way, the motion trend factors at five consecutive sampling moments are calculated to form the motion trend set of all tracks; Step 5.1.4: Solve the variance value of the motion trend set of all tracks, and find the track with the smallest variance value in each type of track as the environmental state value, and the category corresponds to the number of targets.
10. A method for target tracking and data interconnection based on reinforcement learning according to claim 1, characterized in that, Step 5.3 specifically includes the following sub-steps: Step 5.3.1: According to the magnitude of the correlation probability value, select the track with the largest probability value for each environmental state as the corresponding action; Step 5.3.2: If there is a "competition" situation where multiple environmental states select the same track, then allocate the track according to the rules of "there is only one track that may originate from the target for the state" and "the interconnection probability between the track and the state is the largest" until all states are associated with tracks. Step 5.3.3: If the number of available tracks for an environmental state is not unique, select the track according to the rules of "each track has only a unique source target" and "for a given target, at most one track has it as the source", and select the track with the highest interconnection probability for each state.