Method and device for constructing agent facing low earth orbit satellite opportunity epoch dynamic empowerment

Abstract

The application discloses a low-orbit satellite opportunity epoch dynamic weighting-oriented agent construction method and device, which comprises the following steps: obtaining the state space of each epoch by extracting the features of each epoch in the epoch set; determining the initial position of the receiver based on the former epoch in the M continuous epochs in the epoch set; determining the real-time estimated position of the receiver based on the latter epoch; determining the weight action of each epoch in the latter epoch by the agent based on the state space of each epoch and the real position of the receiver in each epoch on the basis of the Markov decision process model; obtaining the Markov decision process tuple according to the designed Markov decision process model; training the dynamic weighting agent by using the tuple; and finally determining the weight of the current epoch by using the trained agent and the state space of the current epoch, and then calculating the real-time estimated position of the receiver, so that the positioning precision and robustness of the low-orbit satellite opportunity signal are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence and satellite navigation and positioning technology, and particularly relates to an intelligent agent construction method and device for low-orbit satellite opportunity epoch dynamic weighting. BACKGROUND

[0002] A global navigation satellite system (GNSS) can provide good positioning, navigation and timing (PNT) services in an open environment, but the signal has high path loss in the transmission process, resulting in low power level at the receiving end. This defect makes the GNSS positioning performance seriously decline or even fail in scenes with physical obstructions such as urban dense areas, dense forest areas, underground tunnels and indoor environments.

[0003] To overcome the above limitations, low earth orbit (LEO) satellite opportunity signals have a signal strength improvement of more than 20 dB due to their orbital height of only 1 / 20~1 / 40 of GNSS satellites, showing stronger penetration ability and higher coverage density, and becoming an effective supplement to GNSS. However, existing real-time positioning algorithms based on iteration (such as Kalman filtering and recursive least squares) usually use fixed weights or simple heuristic rules to homogenously process epochs, and do not fully consider the multi-dimensional characteristics (such as Doppler frequency, signal-to-noise ratio, satellite-ground distance, azimuth angle, etc.) of satellites, epochs and signals, resulting in accumulation of poor epochs and restricting the improvement of positioning accuracy.

[0004] The real-time positioning method in the related art mostly uses fixed weights or simple heuristic rules to process the reliability of epochs, for example, some algorithms only correct the orbit error based on the azimuth angle and the elevation angle, the LEO-NNPON architecture only optimizes the orbit extrapolation accuracy, and the geometric dilution precision (GDOP) selection strategy is limited to a single geometric dilution precision for satellite selection. These methods cannot fully utilize the multi-dimensional characteristics of epochs for dynamic weighting, and are difficult to adapt to positioning needs in complex environments. Therefore, an intelligent method is needed that can comprehensively evaluate the reliability of epochs and dynamically adjust the weights to improve the accuracy and robustness of low-orbit satellite opportunity signal positioning. SUMMARY

[0005] To solve the above technical problems, the application provides an agent construction method and device for low-orbit satellite opportunity epoch dynamic weighting, solves the key technical problems of epoch quality dynamic evaluation and weighting in low-orbit satellite opportunity signal positioning through agent modeling and deep reinforcement learning, significantly improves the positioning accuracy, robustness and adaptability, and provides a new idea for real-time navigation and positioning in a non-cooperative environment. In addition, a computer device and a computer readable storage medium are provided to implement the agent construction method.

[0006] To achieve the above object, the technical scheme adopted by the application is as follows:

[0007] In the first aspect, the application provides an agent construction method for low-orbit satellite opportunity epoch dynamic weighting, which comprises the following steps:

[0008] Step 1: Obtain low-orbit non-cooperative signals of N epochs to obtain an epoch set; wherein the low-orbit non-cooperative signals of each epoch correspond to a satellite, and N is a positive integer; the low-orbit non-cooperative signals of N epochs are sorted in the epoch set according to the receiving time sequence;

[0009] Step 2: Extract features of each epoch to obtain a state space corresponding to each epoch; wherein the state space of each epoch at least includes satellite state, epoch state, signal state and positioning state;

[0010] Step 3: Obtain M consecutive epochs in the epoch set, determine the initial position of the receiver based on the first epochs in the M consecutive epochs, and determine the real-time estimated position of the receiver according to the decision recursion of the agent based on the last epochs in the M consecutive epochs to obtain the real-time estimated position of the receiver corresponding to each epoch in the last epochs; wherein M is a positive integer less than or equal to N, and are both positive integers less than M, and the sum of and is equal to M;

[0011] Step 4: Determine the Markov decision process tuple of each epoch in the last epochs based on the state space and the real position by the agent based on the Markov decision process model, train the epoch dynamic weighting agent using the Markov decision process tuple, make the agent learn the epoch dynamic weighting strategy, and obtain a trained agent model;

[0012] Step 5: Determine the current epoch weight based on the state space of the current epoch using the trained epoch dynamic weighting agent, so that the receiver determines the real-time estimated position after the current epoch participates in the calculation based on at least the current epoch weight using an iterative positioning solution algorithm.

[0013] In a second aspect, the present application provides an agent construction device for low-orbit satellite opportunity epoch dynamic weighting, which comprises:

[0014] An acquisition module configured to acquire low-orbit non-cooperative signals of N epochs to obtain an epoch set; wherein each epoch corresponds to a low-orbit non-cooperative signal of a satellite, and N is a positive integer; the low-orbit non-cooperative signals of the N epochs are sorted in the epoch set according to the order of receiving time;

[0015] An extraction module configured to perform feature extraction on each epoch to obtain a state space corresponding to each epoch; wherein the state space of each epoch at least includes a satellite state, an epoch state, a signal state and a positioning state;

[0016] A recursion module configured to acquire M consecutive epochs in the epoch set, determine an initial position of a receiver based on the first epochs in the M consecutive epochs, and determine a real-time estimated position of the receiver according to the decision of an agent based on the last epochs in the M consecutive epochs to obtain a real-time estimated position corresponding to each epoch in the last epochs; wherein M is a positive integer less than or equal to N, and are both positive integers less than M, and and sum up to M;

[0017] A training module configured to determine a Markov decision process tuple of each epoch in the epochs based on the state space and a real position by an agent on the basis of a Markov decision process model, train an epoch dynamic weighting agent using the Markov decision process tuple, make the agent learn an epoch dynamic weighting strategy, and obtain a trained agent model;

[0018] A determination module configured to determine a current epoch weight based on a state space of a current epoch using the trained epoch dynamic weighting agent, so that the receiver determines a real-time estimated position after the current epoch participates in calculation based on at least the current epoch weight using an iterative positioning solution algorithm.

[0019] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor realizes the agent construction method for low-orbit satellite opportunity epoch dynamic weighting described above when executing the computer program.

[0020] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned method for constructing an intelligent agent for low-orbit satellite opportunity epoch dynamic weighting.

[0021] The present application has the following advantages:

[0022] The method and device for constructing an intelligent agent for low-orbit satellite opportunity epoch dynamic weighting provided by the present application can construct an epoch set by using the received epochs, extract the state space of each epoch by extracting the features of each epoch in the epoch set, obtain M consecutive epochs in the epoch set, determine the initial position of the receiver according to the first epoch in the M consecutive epochs, determine the real-time estimated position of the receiver based on the last epoch, and then determine the Markov decision process tuple of each epoch in the M consecutive epochs based on the state space of each epoch and the real position of the receiver in each epoch by the intelligent agent based on the Markov decision process model, train the dynamic weighting intelligent agent using the Markov decision process tuple, and finally determine the weight of the current epoch by using the trained intelligent agent and the state space of the current epoch, thereby obtaining the real-time estimated position of the receiver, providing an intelligent method capable of comprehensively evaluating the credibility of epochs and dynamically adjusting the weights, and improving the accuracy and robustness of low-orbit satellite opportunity signal positioning. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 FIG. 1 is a flowchart of the method for constructing an intelligent agent for low-orbit satellite opportunity epoch dynamic weighting provided by an embodiment of the present application.

[0024] Figure 2 FIG. 2 is a flowchart of the method for determining the Markov decision process tuple of each epoch provided by an embodiment of the present application.

[0025] Figure 3 FIG. 3 is a structural diagram of the dynamic Bayesian network model corresponding to the state transition model of the Markov decision process provided by an embodiment of the present application.

[0026] Figure 4 FIG. 4 is a flowchart of the method for training the epoch dynamic weighting intelligent agent using the tuple provided by an embodiment of the present application.

[0027] Figure 5 FIG. 5 is a flowchart of the single training process of the DDQN model provided by an embodiment of the present application.

[0028] Figure 6 FIG. 6 is a flowchart of another method for constructing an intelligent agent for low-orbit satellite opportunity epoch dynamic weighting provided by an embodiment of the present application. DETAILED DESCRIPTION​​​

[0029] The application will be further described below in conjunction with the accompanying drawings and embodiments.

[0030] Figure 1 is a flowchart of an agent construction method for low-orbit satellite opportunity epoch dynamic weighting provided by an embodiment of the application. As shown in the figure, the method comprises the following steps: Figure 1

[0031] In step 1, low-orbit non-cooperative signals of N epochs are acquired to obtain an epoch set.

[0032] Each epoch corresponds to a low-orbit non-cooperative signal of a satellite, and N is a positive integer; the low-orbit non-cooperative signals of the N epochs are sorted in chronological order in the epoch set.

[0033] In step 2, feature extraction is performed on each epoch to obtain a state space corresponding to each epoch.

[0034] The state space of each epoch at least includes satellite state, epoch state, signal state and positioning state.

[0035] In step 3, M consecutive epochs are acquired in the epoch set, the initial position of the receiver is determined based on the first K epochs in the M consecutive epochs, and the real-time estimated position of the receiver is determined according to the decision recursion of the agent based on the last L epochs in the M consecutive epochs, to obtain the real-time estimated position of the receiver corresponding to each epoch in the last L epochs.

[0036] K and L are positive integers less than M, and the sum of K and L is equal to M.

[0037] In step 4, based on the state space and the real position, the agent determines the Markov decision process tuple of each epoch in the M consecutive epochs based on a Markov decision process model, trains the epoch dynamic weighting agent using the Markov decision process tuple, makes the agent learn the epoch dynamic weighting strategy, and obtains a trained agent model.

[0038] In step 5, the trained epoch dynamic weighting agent is used to determine the weight of the current epoch based on the state space of the current epoch, so that the receiver determines the real-time estimated position after the current epoch participates in the calculation based on at least the weight of the current epoch using an iterative positioning solution algorithm.

[0039] ​​​​​​​​​In some embodiments of the present application, the method can be performed by a server or a terminal device with certain processing capability. In an example, the terminal device can be a receiver.

[0040] In some embodiments of the present application, N low earth orbit non-cooperative signals of epochs can be received first to obtain an epoch set. Each epoch corresponds to a low earth orbit non-cooperative signal of a satellite, and N is a positive integer. The N low earth orbit non-cooperative signals of epochs are sorted in chronological order in the epoch set.

[0041] Taking Iridium signals as an example of low earth orbit non-cooperative signals, a receiver capable of receiving and analyzing Iridium signals can be used to continuously receive a plurality of Iridium signals on the roof of a building without any obstruction for four weeks, and analyze the Iridium signals. Since the time frame broadcasting mechanism of the Iridium system broadcasting channel is time division multiple access, a single beam of each satellite transmits only a single 90ms burst signal in a 4.32s period, so the receiver can usually only capture the signal of one satellite at the same time, and a single satellite provides only one valid observation value in an epoch. Assuming that the number of epochs corresponding to the collected Iridium signals is , the epoch set is , and each epoch corresponds to an Iridium satellite.

[0042] The features of each epoch can be extracted to obtain the state space corresponding to each epoch. The state space of each epoch includes at least satellite state, epoch state, signal state, and positioning state. In an example, the satellite state can include age of ephemeris, satellite- ground distance, azimuth angle, and elevation angle, the epoch state can include satellite unique identifier and epoch number, the signal state can include signal-to-noise ratio and Doppler frequency, and the positioning state can include generalized GDOP.

[0043] That is, after collecting enough epochs, nine feature parameters of each epoch can be extracted, which are Doppler frequency, signal-to-noise ratio, satellite unique identifier, satellite-ground distance, azimuth angle, and elevation angle, age of ephemeris, epoch number, and generalized GDOP, and stored offline to obtain the state space of each epoch as the object of intelligent agent weighted decision-making, for training a dynamic weighting intelligent agent.

[0044] Before training a dynamically weighted agent, we can first model a Markov Decision Process (MDP) to obtain a MDP tuple, which serves as the training basis for the dynamically weighted agent. This is because the MDP provides a standard mathematical model describing the interaction between the agent and its environment, explicitly modeling the agent's behavior as a process of choosing an action to change the environmental state and obtain a reward after perceiving the environmental state. Modeling the MDP helps the agent understand the epoch-making weighting problem and formulate a strategy, facilitating training using deep reinforcement learning algorithms. The design of the MDP directly affects the agent's learning performance on the epoch-making weighting problem during the training phase.

[0045] In some embodiments of the present invention, M consecutive epochs can be obtained from the epoch set, based on the first M consecutive epochs. The initial position of the receiver is determined by each epoch, and based on the later position in M ​​consecutive epochs... Each epoch recursively determines the receiver's real-time predicted position based on the agent's decisions, thus obtaining the receiver's position in the later stages. The real-time estimated position corresponds to each epoch in the N epochs. Where M is a positive integer less than or equal to N. and All are positive integers less than M, and and The sum of them equals M.

[0046] In other words, the epoch set can be Sort the data according to the order of receipt, and then select consecutive data from it. Each epoch serves as the weighting object for a complete iteration of positioning, where... ,forward Individual calendar The weight is set to 1 by default and is used for the initial localization solution. Individual calendar The weights are determined by the agent, and iterative localization is performed in conjunction with an iterative localization algorithm.

[0047] In some embodiments of the present invention, the location can be determined by an agent based on the state space and the actual position, using a Markov decision process model. The Markov decision process tuple for each epoch is used to train the epoch-dynamically weighted agent, enabling it to learn the epoch-dynamically weighted strategy and obtain a trained agent model.

[0048] Furthermore, the trained epoch-weighted agent can be used to determine the weight of the current epoch based on the state space of the current epoch, so that the receiver can use the iterative positioning algorithm to determine the real-time estimated position of the current epoch after participating in the solution, at least based on the weight of the current epoch.

[0049] Using the technical solutions provided by the embodiments of the present application, by utilizing the received epochs to construct an epoch set, the state space of each epoch in the epoch set is obtained by feature extraction, M consecutive epochs in the epoch set are obtained, the initial position of the receiver is determined according to the first M-1 epochs, the real-time estimated position of the receiver is determined based on the last M-1 epochs, then the Markov decision process tuple of each epoch in the M consecutive epochs is determined by the agent based on the state space of each epoch and the real position of the receiver in each epoch based on the Markov decision process model, the dynamic weighting agent is trained using the Markov decision process tuple, and finally the current epoch weight is determined using the trained agent and the state space of the current epoch, and the real-time estimated position of the receiver is obtained by solving, which provides an intelligent method that can comprehensively evaluate the credibility of the epoch and dynamically adjust the weight, and improves the accuracy and robustness of the low-orbit satellite opportunity signal positioning. In some embodiments of the present application, the Markov decision process tuple of each epoch in the M consecutive epochs can include an environment observation state, a weight action, a transition state and a reward function.

[0050] The environment observation state can include the satellite state, the epoch state and the signal state observed by the epoch dynamic weighting agent at time step t, and the positioning state determined by the epoch dynamic weighting agent at time step t based on the iterative positioning solving algorithm.

[0051] The weight action can include the epoch weight action determined by the epoch dynamic weighting agent based on the environment observation space of the tth epoch. The tth epoch is the epoch corresponding to the time step t.

[0052] The transition state can include the state to which the epoch dynamic weighting agent is transferred after performing the epoch weight action, and the reward function includes the reward obtained by the epoch dynamic weighting agent after completing the transition from the environment observation state of the tth epoch to the transition state through the weight action.

[0053] Further, the weight action of the tth epoch is determined in the following manner: first, an action space is constructed; the action space includes all candidate actions of the tth epoch, and each candidate action corresponds to an epoch weight. Then, the decision strategy of the epoch dynamic weighting agent is used to determine the weight action of the tth epoch based on the environment observation state of the tth epoch.

[0054]

[0055] ​​​​​Meanwhile, the reward function for each epoch can be determined as follows: the initial positioning error is determined based on the difference between the initial position and the receiver's true initial position; the initial position is determined by the receiver using a positioning algorithm based on the previous position. The epoch is determined; the weighted action for the epoch is determined based on the environmental observation state in the epoch tuple; the iterative positioning algorithm is executed based on the weighted action for the epoch to obtain the real-time estimated position of the receiver at the epoch; the positioning error for the epoch is determined based on the real-time estimated position and the actual position of the receiver at the epoch; the formula is used... The rewards for each epoch are calculated sequentially, starting from the first epoch; among them Let be the reward function for the t-th epoch. The reward coefficient related to the change in positioning error. The reward function is related to the percentage change in positioning error. Let be the positioning error obtained by the iterative positioning algorithm at the (t-1)th epoch. The positioning error is obtained by the iterative positioning algorithm at the t-th epoch.

[0056] In other words, defined at time step Within, the intelligent agent learns from the environment (i.e., The system obtains its state and makes weighting decisions within the epochs (and iterative localization algorithms), completes an interaction with the environment, and then... After several time steps, the final localization result is obtained, completing one full iterative localization experiment. In fact, At any time step The intelligent agent is based on the historical period. The Markov decision model corresponding to the decision process of weights can be represented as a tuple. .

[0057] in, Is the intelligent agent at time step The state obtained by observing the environment is represented as ,in, , and From the extracted epoch Feature acquisition, It is derived through an iterative localization algorithm. For the agent to observe the state And take action The state after transition is represented as .

[0058] Is the intelligent agent at time step The observed satellite status is represented as where, is the epoch observed by the agent at time step corresponding to the satellite, is the corresponding ephemeris age, is the epoch observed by the agent at time step corresponding to the satellite, is the azimuth angle of the satellite, is the epoch observed by the agent at time step corresponding to the satellite, is the elevation angle of the satellite, is the epoch observed by the agent at time step corresponding to the satellite, is the slant range of the satellite.

[0059] is the epoch state observed by the agent at time step is represented as where, is the epoch observed by the agent at time step is the epoch number of the epoch is the unique identifier of the satellite,

[0060] is the signal state observed by the agent at time step is represented as where, is the signal-to-noise ratio of the signal observed by the agent at time step is the Doppler frequency of the signal observed by the agent at time step

[0061] Furthermore, is the latest GDOP derived from the iterative-based positioning solution algorithm at time step in order for the agent to observe the current positioning state.

[0062] is the epoch weight action taken by the agent after observing at time step is represented as with 20 possible actions. The agent at time step according to ​​​​​​​After evaluating each state component of the epoch, the credibility of the epoch is judged and quantified as a discrete weight action value, and then the selected weight action is output to the iterative positioning algorithm, which adjusts the iteration coefficient related to the epoch according to the weight action, and finally performs an iterative positioning combined with the information of the epoch.

[0063] is the state of the agent at time step selects After completing the weight decision, the agent transfers from to The reward obtained is represented as wherein, is the state of the agent at time step The positioning error obtained by comparing the positioning result obtained by the iterative positioning algorithm combined with the current epoch information and the epoch weight with the true position, is the positioning error obtained by comparing the initial positioning result obtained by solving by any positioning algorithm with the true position, is the reward coefficient related to the positioning error change value, is the reward coefficient related to the percentage change in positioning error.

[0064] Figure 2 is the flowchart of the method for determining the Markov decision process tuple of each epoch provided by the embodiment of the application. As shown in Figure 2 , the method comprises the following steps:

[0065] In step 411, the environment observation state of the first epoch is obtained; the first epoch is the epoch corresponding to the first time step.

[0066] In step 412, the weight action of the first epoch is determined using the epoch dynamic weighting agent decision strategy.

[0067] In step 413, the iterative positioning algorithm is executed based on the weight action of the first epoch to obtain the real-time estimated position of the receiver at the first epoch.

[0068] In step 414, the reward function of the first epoch is determined based on at least the real-time estimated position of the first epoch.

[0069] In step 415, the environment observation state of the second epoch is determined as the transition state of the first epoch.

[0070] In step 416, the Markov decision process tuple of the first epoch is determined based on the environment observation state, weight action, reward function and transition state of the first epoch.

[0071] In step 417, the above steps S411 to S416 are iteratively performed to determine the Markov decision process tuple of the second epoch to the Tth epoch in turn.

[0072] Wherein the jth epoch is the epoch corresponding to the jth time step, j is greater than or equal to 2 and less than or equal to T.

[0073] In some embodiments of the present application, the Markov decision process is used to determine the Markov decision process tuple of each of the epochs based on the state space, the initial position and the real-time position. The Markov decision process tuple of each of the epochs can be, first, obtaining the environment observation state of the first epoch; the first epoch is the epoch corresponding to the first time step. Then the decision strategy of the agent is determined using the epoch dynamic weighting to determine the weight action of the first epoch, and the iterative positioning algorithm is executed based on the weight action of the first epoch to obtain the real-time estimated position of the receiver in the first epoch. Next, at least based on the real-time estimated position of the first epoch, the reward function of the first epoch is determined, the environment observation state of the second epoch is determined as the transition state of the first epoch, and the Markov decision process tuple of the first epoch is determined based on the environment observation state, the weight action, the reward function and the transition state of the first epoch. Finally, the above steps are iteratively performed to determine the Markov decision process tuple of the second epoch to the Tth epoch in turn.

[0074] Figure 3 is a structural diagram of a dynamic Bayesian network model corresponding to the state transition model of the Markov decision process provided by the embodiments of the present application. As shown in Figure 3 , the state transition process of the Markov decision process can be, first, the agent extracts information from the first epoch signal to obtain signal state and epoch state , and obtains satellite state by comparing TLE ephemeris, then reads the current positioning state from the iterative positioning algorithm, and combines the above obtained state components to obtain state . The agent obtains action after making a decision based on state , and the iterative positioning algorithm will perform an iterative positioning based on action and the information of the first epoch, and obtain positioning error after the positioning result is obtained and the difference is taken with the true position, then is calculated based on the initial positioning error , and is obtained by extracting state components from the second epoch, TLE ephemeris and the positioning algorithm, is taken as , and a single state transition process is completed.

[0075] The state transition is performed in sequence with reference to the above flow until the final positioning result is obtained. After the last action and the corresponding reward are obtained, since the weights of all epochs of this iteration positioning have been determined, the state should be transferred to the termination state .

[0076] The Markov decision process corresponding to the epoch weight determination problem of the low-orbit satellite opportunity signal can be obtained through the above modeling, and the training method of the intelligent agent must be based on the Markov decision process.

[0077] In some embodiments of the present application, the epoch dynamic weight intelligent agent can be a deep reinforcement learning model intelligent agent, and the deep reinforcement learning model at least includes an online network and a target network, wherein the online network is used to determine the estimated Q value, and the target network is used to determine the target Q value.

[0078] In some embodiments, the online network and the target network both include a multilayer perceptron, and the multilayer perceptron includes an input layer, a hidden layer and an output layer. The number of neurons in the input layer is the same as the feature dimension of the state space; the hidden layer includes a first hidden layer and a second hidden layer; the first hidden layer is a fully connected layer, includes neurons, and uses a rectified linear unit (ReLU) as an activation function, and the second hidden layer is also a fully connected layer, includes neurons, and also uses a rectified linear unit (ReLU) as an activation function; the output layer is a fully connected layer, and the number of neurons is the same as the number of discrete actions in the preset action space; each action in the preset action space corresponds to an epoch weight.

[0079] Taking a double deep Q-network (DDQN) model as an example, the network architecture of the deep reinforcement learning model can be constructed in the following way: since the state of the proposed Markov decision process model contains 9 key feature parameters, the state is actually represented as a nine-dimensional feature vector. In addition, the nine features in the vector need to be scaled so that they are all in the same order of magnitude, so that the intelligent agent does not focus on the features with larger values.

[0080] Therefore, the DDQN model provided by this invention can use a multi-layer perceptron (MLP) as an approximator for the Q-value function of DDQN. The online network and the target network have the same network structure. The network structure of this MLP is defined as follows: First, the input layer has 9 neurons, consistent with the dimension of the state feature vector. Second, the network has two hidden layers. The first hidden layer is a fully connected (FC) layer. The layer contains 1 neuron whose activation function is the Rectified Linear Unit (ReLU). The second hidden layer is also a fully connected (FC) layer. There are 20 neurons, and the activation function is ReLU. Finally, the output layer is a fully connected (FC) layer with 20 neurons. Each neuron corresponds one-to-one with a discrete action in the action space of (2) (the first neuron corresponds to action 0.1, the second neuron corresponds to action 0.2, ..., the 20th neuron corresponds to action 2.0). There is no activation function, and the output is the original Q-value estimate.

[0081] Figure 4 This is a flowchart illustrating the method for training a dynamically weighted agent using tuples, as provided in an embodiment of the present invention. Figure 4 As shown, the method includes the following steps:

[0082] In step 421, the epoch set is trimmed to obtain... Each epochal element subset.

[0083] in This is the floor symbol.

[0084] In step 422, an initial exploration rate is determined in the first time step, and a Markov decision process is used based on the initial exploration rate to determine... The first epoch tuple of the first epoch subset is selected from the epoch subsets, and the selected tuple is saved to the replay buffer.

[0085] In step 423, uniform random sampling is performed in the self-playback buffer. We obtain training data from tuples.

[0086] The training data is ,in For training data, The environmental observation state of the i-th tuple. The weight action for the i-th tuple, Let i be the reward function for the i-th tuple. This represents the transition state of the i-th tuple. For the termination flag, when is the termination state is 1, otherwise 0.

[0087] In step 424, the training data is input into the online network and the target network of the DDQN model respectively to obtain the estimated Q value and the target Q value of the training data.

[0088] In step 425, the loss function of the DDQN model is determined based on the estimated Q value and the target Q value, and the network parameters of the online network are updated based on the loss function, and the network parameters of the target network are soft updated.

[0089] In step 426, the initial exploration rate is reduced to obtain the updated exploration rate, and each sub-set is iterated in sequence until all iteration positioning experiments are traversed.

[0090] wherein each sub-set is , and k is the sub-set number corresponding to the current iteration.

[0091] In step 427, each time step is traversed in sequence until all time steps are traversed to obtain the trained epoch dynamic weighting agent.

[0092] In some embodiments of the present application, training the epoch dynamic weighting agent using tuples can be, first, the epoch set is cropped to obtain epoch sub-sets; wherein is the floor symbol. Then, uniformly and randomly sample tuples from the replay buffer to obtain training data , wherein is the training data, is the environment observation state of the i-th tuple, is the weight action of the i-th tuple, is the reward function of the i-th tuple, is the transition state of the i-th tuple, is the termination flag, and when is the termination state is 1, otherwise 0.

[0093] Next, the training data is input into the online network and the target network of the DDQN model respectively to obtain the estimated Q value and the target Q value of the training data, and the loss function of the DDQN model is determined based on the estimated Q value and the target Q value, and the network parameters of the online network are updated based on the loss function, and the network parameters of the target network are soft updated.

[0094] Then, the initial exploration rate is reduced to obtain the updated exploration rate, and each sub-set is iterated in sequence , until all iteration positioning experiments are traversed; wherein k is the sub-set serial number corresponding to the current iteration. Finally, sequentially traverse each time step , until all time steps are traversed, to obtain the trained epoch dynamic weighting agent.

[0095] In some embodiments of the present application, when the iteration is to the kth sub-set of the tth time step during the training of the agent, the tuple of each epoch in the kth sub-set is determined based on the exploration rate using the Markov decision process model . .

[0096] If , set , and ; wherein is the transition state of the tth time step in the kth sub-set environment, is the terminal state, is the terminal flag of the tth time step in the kth sub-set environment. Conversely, if , set , and ; wherein is the transition state of the t+1th time step in the kth sub-set environment.

[0097] Then, a random uniform sampling is performed in the interval [0, 1], and when the sampling result is less than or equal to the exploration rate, the weight action of the tth time step in the kth sub-set environment is randomly selected in the preset action space , and when the sampling result is greater than the exploration rate, the is input into the online network of the DDQN deep reinforcement learning model, and the action corresponding to the neuron with the maximum Q value output by the online network is determined as .

[0098] Then, the epoch weight of the tth time step in the kth sub-set environment is determined based on , the real-time estimated position of the receiver in the tth time step in the kth sub-set environment is determined using the iterative positioning solution algorithm based on the epoch weight, and the reward function of the tth time step in the kth sub-set environment is determined based on the real-time estimated position and the true position . Finally, the epoch tuple when the iteration is to the kth sub-set of the tth time step is determined based on , , , and .

[0099] Figure 5 is the flowchart of the single training process of the DDQN model provided by the embodiments of the present application. As Figure 5As shown, the agent obtains the state from the environment. Then according to Strategies are used to make decisions. Actions are then taken. The weight of the current epoch can then be determined. The iterative localization algorithm performs an iteration based on this weight and the epoch information, and then compares the localization result with the true location to obtain the reward for this decision. Then read the information from the next epoch to obtain the next state. ,according to The key to determining this decision The experience tuples generated during this interaction process Store the data in the playback buffer. Sample the playback buffer to obtain small batches of data. and will The inputs are fed into the online network and the target network respectively to calculate the estimated Q-value and the target Q-value, and then the optimizer... Calculate the loss function based on the estimated Q-value and the target Q-value. The gradient of the online network is obtained by backpropagation, and the parameters of the online network are updated. Finally, the parameters of the target network are updated using a soft update. Complete one training session.

[0100] Furthermore, the complete training process for the DDQN model agent can be a set of epochs. In terms of length Cut at intervals to obtain A subset of epochs, in which... This indicates rounding down, and then rounding the previous values ​​in each epoch subset. Initial positioning is calculated for each epoch, and each subset of epochs corresponds to one operation of length [length missing]. A complete iterative localization experiment was conducted, training the agent on these subsets of epochs. The training steps included:

[0101] Step 1: Iterate through each time step in sequence .

[0102] Step 2: Traverse each complete iterative localization experiment in sequence. .

[0103] Step 3: The agent starts from the current experiment The current time step is observed in the environment. status .

[0104] Step 4: If but .

[0105] Step 5: If then , else .

[0106] Step 6: Randomly and uniformly sample between 0 and 1, if the sample value is less than or equal to , then randomly select an action from the action space as , otherwise, input the state into the online network, select the action corresponding to the neuron with the maximum Q value of the output layer .

[0107] Step 7: input the corresponding weight of into the corresponding iterative-based positioning solution algorithm of the experiment , and calculate the reward obtained by this decision after obtaining the positioning error .

[0108] Step 8: store the experience tuple generated in this interaction process to the replay buffer .

[0109] Step 9: randomly sample from the replay buffer to obtain a small batch of data .

[0110] Step 10: input the small batch of data into the online network and the target network to obtain the estimated Q value and the target Q value of the small batch of data , respectively.

[0111] Step 11: calculate the loss function according to the estimated Q value and the target Q value, and use the Adam optimizer to update the online network by backpropagation.

[0112] Step 12: soft update the parameters of the target network.

[0113] Step 13: reduce the exploration rate: .

[0114] Step 14: return to Step 2 until all experiments are traversed.

[0115] Step 15: return to Step 1 until all time steps are traversed.

[0116] After completing the above training process, the intelligent agent that can evaluate the reliability of ephemeris, distinguish the quality of ephemeris, and realize the dynamic weighting of low-orbit satellite opportunity signal ephemeris is obtained. ​

[0117] In some embodiments of the present application, the exploration rate in the iteration process can be updated in the following way: using the formula updating the exploration rate, wherein is the exploration rate, is the minimum exploration rate, is the decay rate.

[0118] In some embodiments of the present application, the loss function can be determined in the following way: using the formula determining the loss function; wherein is the loss function, is the set of samples from the replay buffer, taking the average after some uniform operation, is the online network's value estimate of the current weight action, s is the current environment observation state, and a is the current weight action, is the network parameter of the online network, is the target Q value, , is the reward obtained by the agent in the current environment observation state through the current weight action, is the discount factor, is the termination flag corresponding to the current state, is the value estimate of the next optimal action by the target network, is the function of determining the value of the independent variable corresponding to the maximum function value, is the weight action selected by the online network after evaluating the value of all candidate actions in the next environment observation state, that is, the optimal next action according to the online network, but it may not be the action actually executed in the next state (because the next step may be exploration), is the transition state, is the candidate weight action corresponding to the next environment observation state, is the network parameter of the target network.

[0119] Further, updating the network parameters of the online network and the network parameters of the target network based on the loss function can be: first, using the adaptive matrix estimation Adam optimizer to calculate the gradient of the online network based on the loss function. Then update . Finally, using the formula update ; wherein is the update rate, which is used to control the speed at which the network parameters of the target network converge to the network parameters of the online network.

[0120] That is, in order to carry out efficient training, the present application simultaneously adopts three training mechanisms—experience replay, soft update and strategy:

[0121] Experience tuples obtained from each interaction of the agent with the environment will be stored in a fixed-size replay buffer , where is the current state , is the next state transferred to by the current action , is a flag (1 if is the terminal state, otherwise 0). The buffer is a First-in First-out (FIFO) queue with a maximum capacity of . When the buffer is full, the oldest experience will be discarded to store new experience. At each training iteration, a mini-batch of experience tuples is uniformly and randomly sampled from the replay buffer to compute the loss function and update the online network.

[0122] The parameters of the target network are not directly updated by gradient descent, but are updated in a soft manner: after updating the parameters of the online network , the parameters of the target network are updated as , where is the update rate, controlling the speed of the target network parameters converging to the online network parameters.

[0123] The present invention adopts policy as the behavior policy to select actions to interact with the environment: with probability , a random action is uniformly selected from the action space ; with probability , the current optimal action corresponding to the maximum Q value output by the online network is selected. During the training process, the exploration rate decays over time, where is the minimum exploration rate, , is the decay rate, , and the initial value of

[0124] , the update of the online network parameters can be, the optimization goal of the online network is to make its predicted Q value as close as possible to the target Q value calculated by the Q value of the target network . The present invention adopts mean squared error as the loss function: ​and uses an Adaptive Moment Estimation (Adam) optimizer based on the loss function Calculate the gradient of the online network, update the parameters of the online network . The learning rate of the Adam optimizer is set to , which is used to control the step size of parameter update.

[0125] Figure 6 is another flowchart of the method for constructing an agent for low-orbit satellite opportunity epoch dynamic weighting provided by the embodiment of the application. As shown in Figure 6 , the method can first collect real low-orbit satellite signals, then perform epoch data feature extraction on the satellite signals, next design a Markov decision process, and construct a deep reinforcement learning algorithm based on the designed Markov decision process, train the agent based on the deep reinforcement learning algorithm, and thus obtain an agent capable of dynamically weighting low-orbit satellite opportunity epochs.

[0126] The technical solution provided by the embodiment of the application, under the premise of using an iterative positioning solution algorithm, designs a Markov decision process to guide the agent to understand the epoch weighting problem, and uses a deep reinforcement learning algorithm to train the agent. Careful selection of epochs and information related to positioning in the algorithm, extraction of the state space observable by the agent, limitation of the epoch weight within a certain range and discretization, consideration of the positioning error change in the early iteration and the positioning error change in the late iteration to design a reasonable reward function, and state transition for each epoch in view of the connection between the previous and subsequent epochs. On the basis of the designed Markov decision process, an efficient training mechanism is adopted, and the deep reinforcement learning algorithm is used to train the agent in a large number of real collected epoch sets.

[0127] The technical solution provided by the embodiment of the application has the following advantages:

[0128] Multi-dimensional feature fusion dynamic weighting: the application extracts nine-dimensional parameters including satellite features (ephemeris age, satellite-ground distance, azimuth angle, elevation angle), epoch features (satellite identifier, epoch number), and signal features (signal-to-noise ratio, Doppler frequency), and a generalized geometric dilution of precision derived from a positioning solution algorithm to construct the state space of the agent, and realizes comprehensive evaluation of the credibility of the epoch. This multi-dimensional feature fusion dynamic weighting mechanism significantly improves the rationality of the weight and the positioning accuracy.

[0129] Intelligent decision-making and adaptive optimization: The application converts the epoch weighting problem into a sequential decision-making problem of an agent through Markov decision process modeling, and trains the agent in combination with deep reinforcement learning (taking double deep Q learning as an example). The agent can dynamically adjust the epoch weight (discrete value of 0.1-2.0) according to the real-time positioning state and epoch characteristics, realize flexible weighting, effectively suppress the influence of poor epochs, and improve the stability and convergence speed of the positioning result.

[0130] End-to-end global optimization capability: The agent designed in the application deeply participates in the whole process of positioning calculation, from epoch feature extraction to weight decision-making, to iterative positioning, forming a closed-loop optimization. Through the design of the reward function (combining the positioning error change value and percentage), the agent can learn the long-term optimal strategy and avoid local optimization, thereby realizing higher positioning accuracy in complex scenarios.

[0131] Wide applicability and scalability: The method does not depend on specific satellite systems or signal types, and can be adapted to different low-orbit satellite constellations (such as Iridium, Starlink, etc.) by adjusting the feature parameters and action space. In addition, the training framework of the agent (such as network structure, reward mechanism) can be flexibly extended according to actual needs, providing a technical foundation for future multi-source fusion positioning.

[0132] On the other hand, the application provides an agent construction device for dynamic weighting of low-orbit satellite opportunity epochs, which comprises:

[0133] An acquisition module configured to acquire low-orbit non-cooperative signals of N epochs to obtain an epoch set; wherein each epoch corresponds to a low-orbit non-cooperative signal of a satellite, and N is a positive integer; the low-orbit non-cooperative signals of the N epochs are sorted in the epoch set according to the order of reception time;

[0134] An extraction module configured to perform feature extraction on each epoch to obtain a state space corresponding to each epoch; wherein the state space of each epoch at least includes satellite state, epoch state, signal state and positioning state;

[0135] A recursion module configured to acquire M consecutive epochs in the constructed epoch set, determine the initial position of the receiver based on the first epochs in the M consecutive epochs, and determine the real-time estimated position of the receiver based on the decision of the agent according to the last epochs in the M consecutive epochs to obtain the real-time estimated position of the receiver corresponding to each epoch in the constructed last epochs; wherein M is a positive integer less than or equal to N, and are positive integers less than M, and and the sum of which is equal to M;

[0136] a training module configured to determine, by the agent, the construction a Markov decision process tuple of each of the epochs, using the construction Markov decision process tuple to train the epoch dynamic weighting agent to learn an epoch dynamic weighting strategy, to obtain a trained agent model;

[0137] a determination module configured to determine, by the trained epoch dynamic weighting agent, a current epoch weight based on a state space of a current epoch, so that the construction receiver determines a real-time estimated position after participating in calculation of the current epoch based on the construction current epoch weight using an iterative positioning solution algorithm.

[0138] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned agent construction method for low-orbit satellite opportunity epoch dynamic weighting when executing the computer program.

[0139] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer is executed by the processor to implement the above-mentioned agent construction method for low-orbit satellite opportunity epoch dynamic weighting.

[0140] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above are only specific embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing an agent oriented to low earth orbit satellite opportunity epoch dynamic empowerment, characterized in that, The method comprises: Step 1, obtaining low-orbit non-cooperative signals of N epochs to obtain an epoch set; wherein each epoch corresponds to a low-orbit non-cooperative signal of a satellite, and N is a positive integer; the low-orbit non-cooperative signals of the N epochs are sorted in the epoch set according to the order of receiving time; Step 2, performing feature extraction on each epoch to obtain a state space corresponding to each epoch; wherein the state space of each epoch at least includes satellite state, epoch state, signal state and positioning state; Step 3: Obtain M consecutive epochs from the epoch set, and based on the first M consecutive epochs... The initial position of the receiver is determined by each epoch, and based on the subsequent epochs of the M consecutive epochs... Each epoch determines the receiver's real-time estimated position based on the agent's decision recursion, thus obtaining the receiver's position in the subsequent... The real-time estimated position corresponds to each epoch in the N epochs; where M is a positive integer less than or equal to N. and All are positive integers less than M, and and The sum equals M; Step 4, determining the Markov decision process tuple of each epoch in the state space and the real position based on the Markov decision process model by the agent epoch dynamic weighting strategy, obtaining a trained agent model; Step 5, using the trained epoch dynamic weighting agent to determine the current epoch weight based on the state space of the current epoch, so that the receiver determines the real-time estimated position after the current epoch participates in the calculation based on at least the current epoch weight by using an iterative positioning calculation algorithm.

2. The method of claim 1, wherein, The Markov decision process tuple for each epoch of the epochs includes an environment observation state, a weighted action, a transition state, and a reward function; The environment observation state comprises a satellite state, an epoch state and a signal state observed by the epoch dynamic empowerment agent at a time step t, and a positioning state determined by the epoch dynamic empowerment agent at the time step t based on an iterative positioning calculation algorithm. ; The weight action includes an epoch weight action determined by the epoch dynamic weighting agent based on the environment observation space of the tth epoch; the tth epoch is an epoch corresponding to a time step t; The transition state includes a state to which the epoch dynamic weighting agent is transferred after performing the epoch weight action, and the reward function includes a reward obtained by the epoch dynamic weighting agent after completing the environment observation state of the tth epoch by transferring to the transition state through the weight action.

3. The method of claim 2, wherein, The weight action of the tth epoch is determined in the following way: An action space is constructed; the action space includes all candidate actions of the tth epoch, and each candidate action corresponds to an epoch weight; The decision strategy of the epoch dynamic weighting agent is used to determine the weight action of the tth epoch based on the environment observation state of the tth epoch.

4. The method of claim 2, wherein, The reward function of each epoch is determined in the following way: The initial positioning error is determined based on the difference between the initial position and the receiver's true initial position; the initial position is determined by the receiver using a positioning algorithm based on the previous position. Each epoch is determined; The weight action of the tth epoch is determined based on the environment observation state in the tth epoch tuple; An iterative positioning calculation algorithm is executed based on the weight action of the tth epoch to obtain the real-time estimated position of the receiver at the tth epoch; The positioning error of the tth epoch is determined based on the real-time estimated position and the true position of the receiver at the tth epoch; The reward of each epoch is calculated in turn from the first epoch using the formula where is the reward function of the tth epoch, is the reward coefficient related to the positioning error change value, is the reward function related to the positioning error change percentage, is the positioning error obtained by the iterative positioning solution algorithm at the t-1th epoch, is the positioning error obtained by the iterative positioning solution algorithm at the tth epoch.

5. The method of claim 2, wherein, The using Markov decision process determines the Markov decision process tuple for each of the epochs, including: The environment observation state of the first epoch is obtained; the first epoch is an epoch corresponding to a first time step; The weight action of the first epoch is determined using the decision strategy of the epoch dynamic weighting agent; An iterative positioning calculation algorithm is executed based on the weight action of the first epoch to obtain the real-time estimated position of the receiver at the first epoch; The reward function of the first epoch is determined based at least on the real-time estimated position of the first epoch; The environment observation state of the second epoch is determined as the transition state of the first epoch; The Markov decision process tuple of the first epoch is determined based on the environment observation state, weight action, reward function and transition state of the first epoch; The above steps are iteratively executed to sequentially determine the Markov decision process tuple of the jth epoch, wherein the jth epoch is an epoch corresponding to a jth time step, and j is greater than or equal to 2 and less than or equal to T.

6. The method of claim 2, wherein, The epoch dynamic weighting agent is a deep reinforcement learning model agent; The deep reinforcement learning model at least includes an online network and a target network; the online network is used to determine an estimated Q value, and the target network is used to determine a target Q value; The online network and the target network each comprise a multi-layer perceptron, which comprises an input layer, a hidden layer and an output layer; The number of neurons of the input layer is the same as the feature dimension of the state space; The hidden layers include a first hidden layer and a second hidden layer; the first hidden layer is a fully connected layer and includes neurons, and the second hidden layer is also a fully connected layer and includes neurons. The output layer is a fully connected layer, and the number of neurons of the output layer is the same as the number of discrete actions in a preset action space; each action in the preset action space corresponds to an epoch weight.

7. The method of claim 6, wherein, The training of the epoch dynamic weighting intelligent agent using the tuple comprises: trimming the set of epochs to obtain a subset of epochs; wherein is a floor symbol. In the first time step, an initial exploration rate is determined, and based on this initial exploration rate, a Markov decision process is used to determine the... The first epoch tuple of the first epoch subset is selected from the epoch subsets, and the determined first epoch tuple is saved to a replay buffer with a fixed capacity. Uniform random sampling from a playback buffer a tuple, obtaining training data wherein is training data, is an environment observation state for the i-th tuple, is a weight action for the i-th tuple, is a reward function for the i-th tuple, is a transition state for the i-th tuple, is a termination flag, when is a termination state is 1, otherwise 0; The training data are input into the online network and the target network of the deep reinforcement learning model respectively to obtain estimated Q values and target Q values of the training data; A loss function of the deep reinforcement learning model is determined based on the estimated Q values and the target Q values, and network parameters of the online network are updated based on the loss function, and network parameters of the target network are soft updated; decreasing the initial exploration rate to obtain an updated exploration rate, and iteratively traversing each sub-set in order , sequentially performing the above operations until all iterative positioning experiments are traversed; wherein k is the current corresponding sub-set serial number; traversing the time steps in order The above operations are sequentially performed until all time steps are traversed, obtaining the trained epoch dynamic weighting agent.

8. The method of claim 7, wherein, at the kth subset of the tth time step, determining, based on an exploration rate, the kth subset of the tth time step using a Markov decision process model a tuple for each of the plurality of epochs, comprising: The dynamic empowerment agent observes an environment observation state at time step t from the kth subset of environments ; in response to determining then set , and ; wherein is a transition state in the kthsub-set environment at time step t, is a terminal state, is a terminal flag bit in the kthsub-set environment at time step t; In response to determining then set , and ; wherein is the transition state in the kth subset environment at time step t+1. Uniformly sampling between intervals [0, 1], when the sampling result is less than or equal to the exploration rate, randomly selecting a weight action of time step t in the kth subset of the preset action space in the environment , and when the sampling result is greater than the exploration rate, , inputting an online network of the deep reinforcement learning model, determining the action corresponding to the neuron outputting the maximum Q value of the online network as ; based on determining an epoch weight for time step t in the kth subset environment, determining a real-time estimated position of the receiver at time step t in the kth subset environment based on the epoch weight using an iterative positioning solution algorithm, and determining a reward function for time step t in the kth subset environment based on the real-time estimated position and the true position ; determining an epoch tuple at a kth sub-set iteration to a tth time step based on the , , and ​ 9. The method of claim 7, wherein, The exploration rate is updated in the following manner in the iteration process: Using the formula updating the exploration rate, wherein is the exploration rate, is the minimum exploration rate, is the decay rate.

10. The method of claim 7, wherein, The loss function is determined in the following manner: using the formula determining the loss function; wherein is the loss function, is a set of samples taken from a replay buffer taking an average after some uniform operation, is an online network's value estimate for a current weight action, is a current environment observation state, is a current weight action, is an online network's network parameters, is a target Q value, , is a reward for an agent in a current environment observation state by a current weight action, is a discount factor, is a termination flag corresponding to a current state, is a target network's value estimate for a next optimal action, is a function for determining a value of an independent variable corresponding to a maximum function value, is an online network's weight action selected from value estimates for all candidate actions in a next environment observation state, is a transition state, is a candidate weight action corresponding to a next environment observation state, is a target network's network parameters; The updating of the network parameters of the online network and the network parameters of the target network based on the loss function comprises: An adaptive moment estimation Adam optimizer is used to calculate the gradient of the online network based on the loss function; updating the gradient based on the gradient ; using the formula updating the ; wherein is an update rate for controlling how quickly the network parameters of the target network converge toward the network parameters of the online network.

11. An agent construction device oriented to low earth orbit satellite opportunity epoch dynamic empowerment, characterized by, The apparatus comprises: An acquisition module configured to acquire low-orbit non-cooperative signals of N epochs to obtain an epoch set; each epoch corresponds to a low-orbit non-cooperative signal of a satellite, and N is a positive integer; the low-orbit non-cooperative signals of the N epochs are sorted in chronological order in the epoch set according to the receiving time; An extraction module configured to perform feature extraction on each epoch to obtain a state space corresponding to each epoch; the state space of each epoch at least includes a satellite state, an epoch state, a signal state and a positioning state; The recursive module is configured to obtain M consecutive epochs from the epoch set, and based on the first M consecutive epochs... The initial position of the receiver is determined by each epoch, and based on the subsequent epochs of the M consecutive epochs... Each epoch determines the receiver's real-time estimated position based on the agent's decision recursion, thus obtaining the receiver's position in the subsequent... The real-time estimated position corresponds to each epoch in the N epochs; where M is a positive integer less than or equal to N. and All are positive integers less than M, and and The sum equals M; a training module configured to determine, by an agent, the state space and the real position based on a Markov decision process model, and determine, by the agent, the Markov decision process tuple of each of the epochs, train an epoch dynamic weighting agent using the Markov decision process tuple, make the epoch dynamic weighting agent learn an epoch dynamic weighting strategy, and obtain a trained agent model. A determination module configured to determine an epoch weight of a current epoch based on a state space of the current epoch using the trained epoch dynamic weighting intelligent agent, so that the receiver determines a real-time estimated position after the current epoch participates in the calculation based at least on the epoch weight using an iterative positioning calculation algorithm.

12. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the intelligent agent construction method for low-orbit satellite opportunity epoch dynamic weighting according to any one of claims 1 to 10.

13. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer is executed by the processor to implement the intelligent agent construction method for low-orbit satellite opportunity epoch dynamic weighting according to any one of claims 1 to 10.

Images