A model training method and a track planning method and device for geomagnetic navigation

Abstract

The present specification discloses a model training method and a track planning method and device for geomagnetic navigation. The model training method comprises: obtaining geomagnetic information of a target area and environmental information of a target carrier's motion environment at a first moment and inputting them into a track planning model to be trained, determining the motion decision information of the target carrier through a decision network; determining the state information at a second moment according to the motion decision information and the state information at the first moment; determining a reward value according to the state information at the first moment and the state information at the second moment; inputting the state information at the first moment and the state information at the second moment into a value network to determine, through the value network, an evaluation value corresponding to the state information at the first moment and an evaluation value corresponding to the state information at the second moment; determining a loss value and performing model training according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment and the reward value.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a model training method and a geomagnetic navigation track planning method and device. Background Art

[0002] In recent years, with the development of geomagnetic sensors and geomagnetic field measurement technology, geomagnetic matching navigation technology has received increasing attention. Compared with traditional satellite navigation and inertial navigation, geomagnetic matching navigation has the advantages of no cumulative error, strong anti-interference, good concealment, and rich navigation information. It has good application prospects in deep-sea navigation, underground navigation and other low-light illumination, weak satellite navigation signals, and navigation in sea areas, deserts and other areas with similar optical characteristics.

[0003] However, in the current geomagnetic navigation process, the geomagnetic adaptation area is usually searched and selected based on the manually set geomagnetic statistical characteristics, and then the track is planned in the static geomagnetic adaptation area according to the track planning start and end points and environmental factors. Since this method is too dependent on prior geomagnetic information, it is difficult to quickly and accurately determine the information required for track planning from a single geomagnetic information, resulting in low efficiency and poor accuracy of the overall track planning process.

[0004] Therefore, how to improve the efficiency and accuracy of trajectory planning for geomagnetic navigation is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a model training method and a geomagnetic navigation track planning method and device to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a model training method, including:

[0008] Acquire geomagnetic information of the target area and environmental information of the moving environment of the target carrier at the first moment as the state information at the first moment;

[0009] Inputting the state information at the first moment into the trajectory planning model to be trained, so as to determine the motion decision information of the target carrier through the decision network in the trajectory planning model;

[0010] Determine the state information at a second moment according to the motion decision information and the state information at the first moment, wherein the second moment is a moment after the first moment;

[0011] determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment;

[0012] Inputting the state information at the first moment and the state information at the second moment into the value network of the trajectory planning model, so as to determine the evaluation value corresponding to the state information at the first moment and the evaluation value corresponding to the state information at the second moment through the value network;

[0013] The loss value of the trajectory planning model is determined according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment, and the reward value, and the trajectory planning model is trained according to the loss value.

[0014] Optionally, the environmental information includes at least: location information of the target carrier at the first moment, location information of the target point, obstacle and threat area information, explored area information and unexplored area information.

[0015] Optionally, determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes:

[0016] Determining, according to the state information at the first moment and the state information at the second moment, a geomagnetic adaptation rate corresponding to the motion trajectory of the target carrier from the first moment to the second moment;

[0017] If the geomagnetic matching rate is higher than the average geomagnetic matching rate of the target area, the reward value is increased; otherwise, the reward value is reduced.

[0018] Optionally, determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes:

[0019] Determine the distance of the target carrier relative to the target point at the first moment according to the state information at the first moment, and determine the distance of the target carrier relative to the target point at the second moment according to the state information at the second moment;

[0020] If the distance of the target carrier relative to the target point at the second moment is smaller than the distance of the target carrier relative to the target point at the first moment, the reward value is increased; otherwise, the reward value is decreased.

[0021] Optionally, determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes:

[0022] Determine, according to the state information at the second moment, the distance of the target carrier relative to the center of the influence area and the center of the threat area of ​​the obstacle at the second moment as a determination distance;

[0023] Determining, according to the determination distance, whether the target carrier is located in the obstacle influence area and / or the threat area at the second moment;

[0024] If so, the reward value is decreased.

[0025] Optionally, determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes:

[0026] According to the state information at the first moment and the state information at the second moment, determining whether the target carrier can reach the target point within the maximum range allowed by the remaining fuel of the target carrier;

[0027] If the target carrier can reach the target point within the maximum range allowed by the remaining fuel, the reward value is increased; otherwise, the reward value is reduced.

[0028] Optionally, determining the loss value of the trajectory planning model according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment, and the reward value specifically includes:

[0029] determining an expected evaluation value according to the evaluation value corresponding to the state information at the second moment and the reward value;

[0030] The loss value is determined according to a deviation between the evaluation value corresponding to the state information at the first moment and the expected evaluation value.

[0031] Optionally, training the trajectory planning model according to the loss value specifically includes:

[0032] Taking minimizing the loss value as the optimization goal, the value network of the trajectory planning model is trained, and taking maximizing the expected evaluation value as the optimization goal, the decision network of the trajectory planning model is trained.

[0033] This specification provides a geomagnetic navigation track planning method, including:

[0034] Obtain the geomagnetic information of the area to be planned and the environmental information of the target carrier's motion environment at the current moment;

[0035] Inputting the geomagnetic information of the area to be track planned and the environmental information of the movement environment of the target carrier at the current moment into the track planning model, so as to determine the movement decision information of the target carrier at the next moment through the track planning model, wherein the track planning model is trained by the above-mentioned model training method;

[0036] According to the motion decision information of the target carrier at each moment, the trajectory planning is performed for the target carrier.

[0037] Optionally, before performing track planning for the target carrier according to the motion decision information of the target carrier at each moment, the method further includes:

[0038] Determine a geomagnetic adaptation area in the area to be planned according to the geomagnetic information of the area to be planned and the environmental information;

[0039] According to the motion decision information of the target carrier at each moment, the trajectory planning of the target carrier is carried out, specifically including:

[0040] With the constraint that each track point of the target carrier during its movement is located in the geomagnetic adaptation area, local track planning is performed on the target carrier according to the movement decision information of the target carrier at each moment.

[0041] This specification provides a model training device, including:

[0042] An acquisition module, used to acquire geomagnetic information of a target area and environmental information of a moving environment where a target carrier is located at a first moment, as state information at the first moment;

[0043] A decision module, used for inputting the state information at the first moment into the trajectory planning model to be trained, so as to determine the motion decision information of the target carrier through the decision network in the trajectory planning model;

[0044] a determination module, configured to determine state information at a second moment according to the motion decision information and the state information at the first moment, wherein the second moment is a moment next to the first moment;

[0045] A reward module, used for determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment;

[0046] a value module, used for inputting the state information at the first moment and the state information at the second moment into the value network of the trajectory planning model, so as to determine the evaluation value corresponding to the state information at the first moment and the evaluation value corresponding to the state information at the second moment through the value network;

[0047] A training module is used to determine the loss value of the trajectory planning model according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment and the reward value, and to train the trajectory planning model according to the loss value.

[0048] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned trajectory planning model training method and trajectory planning method when executing the program.

[0049] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0050] In the model training method and trajectory planning method for geomagnetic navigation provided in the present specification, the geomagnetic information of the target area and the environmental information of the motion environment in which the target carrier is located at the first moment are obtained and input into the trajectory planning model to be trained, and the motion decision information of the target carrier is determined through the decision network; the state information at the second moment is determined according to the motion decision information and the state information at the first moment; the reward value is determined according to the state information at the first moment and the state information at the second moment; the state information at the first moment and the state information at the second moment are input into the value network to determine the evaluation value corresponding to the state information at the first moment and the evaluation value corresponding to the state information at the second moment through the value network; the loss value is determined according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment and the reward value, and the model training is performed.

[0051] It can be seen from the above method that this scheme does not rely entirely on geomagnetic information during the training of the trajectory planning model, but combines geomagnetic information and motion environment information to determine the motion decision at each moment. Compared with the method of relying solely on geomagnetic information for geomagnetic navigation, motion environment information can provide an effective reference for the acquisition of motion decision information, thereby accelerating the overall efficiency and accuracy. In addition, this scheme is guided by the evaluation value determined by the value network, and the model is trained by combining the decision network and the value network, which further improves the model performance and more fully guarantees the accuracy of subsequent geomagnetic navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0053] Figure 1 A flowchart of the model training method provided in this manual;

[0054] Figure 2 A schematic diagram of the structure of a decision network provided in this specification;

[0055] Figure 3 A schematic diagram of the structure of a value network provided for this specification;

[0056] Figure 4 A flowchart of a geomagnetic navigation track planning method provided in this specification;

[0057] Figure 5 A schematic diagram of a motion trajectory based on the geomagnetic adaptation zone provided in this specification;

[0058] Figure 6 A schematic diagram of a model training device provided in this specification;

[0059] Figure 7 A schematic diagram of a geomagnetic navigation track planning device provided in this specification;

[0060] Figure 8 A method corresponding to the Figure 1 or Figure 4 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0062] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0063] Figure 1 A flow chart of a model training method provided in this specification includes the following steps:

[0064] S101: Acquire geomagnetic information of a target area and environmental information of a moving environment in which a target carrier is located at a first moment, as state information at the first moment.

[0065] Geomagnetic navigation technology, also known as geomagnetic positioning navigation or magnetic navigation technology, is an advanced method that uses geomagnetic magnetic field information to achieve navigation. The geomagnetic field is the earth's inherent magnetic field, and its intensity and direction vary with different geographical locations. In addition, the geomagnetic field has rich characteristics, including total intensity, vector intensity, magnetic inclination, magnetic declination and intensity gradient. These characteristics provide rich geomagnetic matching positioning information for geomagnetic navigation.

[0066] The basic principle of geomagnetic navigation technology is to use the geomagnetic field as a natural coordinate system, measure the strength and direction of the geomagnetic field, and combine it with the earth's magnetic field model to calculate the position information of the carrier. The geomagnetic field has unique vector characteristics and position correlation, which makes geomagnetic navigation a passive, autonomous and covert navigation method.

[0067] Based on this, this specification provides a model training method and a track planning method for geomagnetic navigation, which integrates sensor information collected in real time and dynamically, and performs fine local track planning within the geomagnetic adaptation area obtained by dynamic search, thereby improving the dynamic nature of track planning, and thus providing a good foundation and conditions for the implementation of long-duration and high-precision geomagnetic matching navigation algorithms.

[0068] In this specification, the execution entity for implementing a model training method and a track planning method for geomagnetic navigation can be a designated device such as a server. Of course, it can also be a target carrier or a client installed on a designated device. For the sake of ease of description, the following will only take the server as an example of the execution entity to illustrate a model training method and a track planning method for geomagnetic navigation provided in this specification.

[0069] In order to efficiently search for the effective magnetic adaptation area from the starting point to the target point during geomagnetic navigation, while taking into account the navigation positioning error compensation and navigation accuracy requirements, the server can use the information collected by multiple sensors as the state information at a known time and input it into the trajectory planning model, and extract the state features through deep convolutional network fusion.

[0070] The server may obtain a priori geomagnetic reference map of the target area and extract geomagnetic information therefrom.

[0071] In this specification, the server can divide the target area covering the starting point and target point of the target carrier (such as unmanned vehicles, unmanned boats, robots and other unmanned equipment) into M×N unit grids. The geomagnetic information is based on the prior geomagnetic reference map of the target area, including multiple geomagnetic characteristic parameters, each geomagnetic characteristic parameter corresponds to a channel, and each channel stores the corresponding geomagnetic data in the form of a two-dimensional array. Specifically, each array element Represented as a cell grid The corresponding geomagnetic characteristic parameter value, wherein the geomagnetic characteristic parameter (geomagnetic information) may include: total magnetic field, vector geomagnetic three-component data, magnetic declination, magnetic inclination, etc. Further, the geomagnetic data may be normalized.

[0072] In addition, the server can also obtain environmental information of the moving environment in which the target carrier is located at the first moment. The environmental information may be real-time information collected by the target carrier through sensors such as radars and cameras during its movement. The above environmental information may include at least: the position information of the target carrier at the first moment, the position information of the target point, obstacle and threat area information, explored area information and unexplored area information.

[0073] In the process of training the geomagnetic model, the first moment can be understood as any moment of a known state (environmental information has been determined) including the initial moment. Except for the initial moment, each time the target carrier executes a motion decision and determines the state information at that moment, that moment can be taken as the first moment, and the next moment of the state information to be determined can be taken as the second moment.

[0074] In this specification, environment and other information can be represented by three channels, each channel uses an M×N two-dimensional array to represent the carrier's current position and target point position information, obstacle threat area information, explored area and unexplored area information respectively.

[0075] Specifically, in the carrier's current position and target point position channels, the array elements corresponding to the grids where the current position and target position are located are represented by 1, and the rest are represented by 0; in the obstacle position and threat area channels, the array elements corresponding to the grids where the obstacle position and threat area are located are represented by 0, and the rest are represented by 1; in the explored area and unexplored area information channels, the array elements corresponding to the grids where the explored area is located are represented by 0, and the unexplored area is represented by 1.

[0076] After determining the geomagnetic information and the environmental information at the first moment (current moment), the server can use the geomagnetic information and environmental information as the status information at the first moment. .

[0077] S102: Inputting the state information at the first moment into the trajectory planning model to be trained, so as to determine the motion decision information of the target carrier through the decision network in the trajectory planning model.

[0078] In this specification, a decision network may be provided in the trajectory planning model. and value network ,in, is the motion decision information output by the decision network at the first moment, are the model parameters of the decision network, are the model parameters of the value network.

[0079] The server will send the status information of the first moment After the trajectory planning model is input, the decision network can , output motion decision information. For ease of understanding, this specification provides a structural diagram of a decision network, such as Figure 2 shown.

[0080] Figure 2 A schematic diagram of the structure of a decision network provided in this specification.

[0081] Among them, the status information After inputting the decision network, the feature vector can be extracted through the convolutional network therein, and then the feature is input into the fully connected network, and the output of the fully connected network is activated through the activation function to obtain the motion decision information.

[0082] The motion decision information may be a probability distribution value of the target carrier in each motion direction. The motion direction may be discrete, including eight motion directions: up, down, left, right, upper left, lower left, upper right, and lower right. The server may extract a motion direction from the probability distribution values ​​of different motion directions as the target motion direction of the target carrier. Of course, the policy network may also directly output the target motion direction as the motion decision information.

[0083] S103: Determine the state information at a second moment according to the motion decision information and the state information at the first moment, wherein the second moment is a next moment of the first moment.

[0084] After determining the motion decision information, the server can determine the environmental information after the target carrier performs the motion according to the target motion direction in the motion decision information under the environmental information at the first moment according to the motion decision information and the state information at the first moment, and use the geomagnetic information and the environmental information after the target carrier performs the motion (the environmental information corresponding to the next moment after the first moment) as the state information corresponding to the second moment .

[0085] S104: Determine a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment.

[0086] Determine the state information corresponding to the second moment After that, the server can use the status information at the first moment And the state information at the second moment, determine the reward value for the motion decision information .

[0087] In this manual, the reward value There are many ways to determine , including:

[0088] The server can use the geomagnetic adaptation rate reward function To determine the reward value, according to the state information at the first moment and the state information at the second moment, determine the geomagnetic adaptation rate corresponding to the motion trajectory of the target carrier from the first moment to the second moment. If the geomagnetic adaptation rate is higher than the average geomagnetic adaptation rate of the entire target area, increase the reward value, otherwise reduce the reward value.

[0089] In this specification, the geomagnetic adaptation rate reward function The geomagnetic matching simulation result can be set based on several generated fine tracks in the m×n grid of the geomagnetic matching search area through geomagnetic matching navigation algorithms such as Mean Square Difference (MSD) or Iterated Closest Contour Point (ICCP). If the geomagnetic matching rate is higher than the average geomagnetic matching rate of the entire navigation area, a reward will be obtained, otherwise the reward value will be reduced, where the geomagnetic matching rate refers to the proportion of the number of tracks whose navigation positioning accuracy meets the requirements of the navigation task in the total number of tracks generated.

[0090] The server can also use the distance reward function To calculate the reward value, determine the distance of the target carrier relative to the target point at the first moment according to the state information at the first moment, and determine the distance of the target carrier relative to the target point at the second moment according to the state information at the second moment. If the distance of the target carrier relative to the target point at the second moment is less than the distance of the target carrier relative to the target point at the first moment, increase the reward value, otherwise reduce the reward value. Reward function It can be expressed as:

[0091]

[0092] in, and are the distances of the target carrier relative to the target point at the first moment and the second moment respectively, is the reward factor. When the carrier approaches the target point, it will be rewarded, otherwise, it will be punished.

[0093] The server can also use the threat area reward function To calculate the reward value, according to the state information at the second moment, determine the distance of the target carrier relative to the center of the obstacle's influence area and the center of the threat area at the second moment as the judgment distance, and then judge whether the target carrier is located in the obstacle's influence area and / or threat area at the second moment based on the judgment distance. If so, reduce the reward value.

[0094] Determine the distance of the target carrier relative to the center of the obstacle threat area at the second moment, and determine whether the target carrier is located in the obstacle threat area at the second moment according to the distance of the target carrier relative to the center of the obstacle threat area at the second moment. If so, reduce the reward value. The reward function It can be expressed as:

[0095]

[0096] in, is the distance between the current position of the target carrier and the center of the obstacle or threat area, is the expansion radius or influence range radius of the obstacle or threat area, When the carrier enters the influence range of obstacles or threat zones, it will be penalized.

[0097] In addition, the server can also use the flight distance reward function To calculate the reward value, based on the state information at the first moment and the state information at the second moment, determine whether the target carrier can reach the target point within the maximum range allowed by the remaining fuel of the target carrier; if the target carrier can reach the target point within the maximum range allowed by the remaining fuel, increase the reward value, otherwise reduce the reward value.

[0098] Furthermore, the server can combine the reward values ​​calculated by all the above reward functions to determine the reward value for the motion decision information. , reward value It can be further expressed as:

[0099]

[0100] Among them, w is the preset weight corresponding to different reward values. The values ​​of each weight are all within (0,1) and satisfy the weight + + + The weight values ​​of each item can be set according to actual conditions, and this specification does not make any specific restrictions on this.

[0101] S105: Inputting the state information at the first moment and the state information at the second moment into the value network of the trajectory planning model to determine, through the value network, an evaluation value corresponding to the state information at the first moment and an evaluation value corresponding to the state information at the second moment.

[0102] After determining the state information at the first moment and the state information at the second moment, the server can input the state information at the first moment and the state information at the second moment into the value network of the trajectory planning model. , to determine the evaluation value corresponding to the state information at the first moment through the value network and the evaluation value corresponding to the state information at the second moment .

[0103] Among them, the evaluation value corresponding to the state information at the first moment is It can be expressed as:

[0104]

[0105] The evaluation value corresponding to the state information at the second moment It can be expressed as:

[0106]

[0107] For each moment of state information, the evaluation value corresponding to the state information is used to represent the expected total reward obtained by taking the possible action at the next moment under the state corresponding to the state information, that is, how valuable the current state is.

[0108] For ease of understanding, this specification provides a structural diagram of a value network, such as Figure 3 shown.

[0109] Figure 3 A structural diagram of a value network provided in this specification.

[0110] Among them, after the server inputs the status information into the value network, the value network can extract features through the deep convolutional network, input the extracted feature vector into the fully connected network, and output the evaluation value corresponding to the status information through the fully connected network.

[0111] S106: Determine a loss value of the trajectory planning model according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment, and the reward value, and train the trajectory planning model according to the loss value.

[0112] Specifically, the server can determine the expected evaluation value based on the evaluation value and reward value corresponding to the state information at the second moment. , expected evaluation value It can be expressed as:

[0113]

[0114] Here, γ is the discount factor of the return, and its specific value can be set according to actual conditions, and this specification does not make any specific limitation on this.

[0115] The server may use the evaluation value corresponding to the state information at the first moment as a predicted value, and thereby determine the loss value of the value network according to the deviation between the evaluation value corresponding to the state information at the first moment and the expected evaluation value. The loss value may be expressed as:

[0116]

[0117] The server can update the network parameters of the value network with minimizing the loss value as the optimization goal, thereby training the value network of the trajectory planning model. The network parameters of the updated value network It can be expressed as:

[0118]

[0119] At the same time, the server can output the expected evaluation value of the value network As a guide, with expected evaluation value Taking maximization as the optimization goal, the decision network of the trajectory planning model is trained so that the action decision information it outputs can determine the maximum expected evaluation value of the value network.

[0120] Specifically, the server can update the network parameters of the policy network , so that the objective function Maximize the objective function It can be expressed as:

[0121]

[0122] The policy gradient can be derived as:

[0123]

[0124] Let the policy gradient be approximated as:

[0125]

[0126] The policy network parameters The update formula is:

[0127]

[0128] Based on this, the network parameters of the updated policy network It can be expressed as:

[0129]

[0130] After completing the training of the trajectory planning model (such as the value network and the policy network converge to a preset range or reach a preset number of training times), the server can deploy it for subsequent geomagnetic navigation tasks.

[0131] This manual further describes a trajectory planning method provided in this manual from the perspective of practical application. Figure 4 shown.

[0132] Figure 4 A schematic diagram of a flow chart of a geomagnetic navigation track planning method provided in this specification includes the following steps:

[0133] S401: Acquire geomagnetic information of the area to be track planned and environmental information of the target carrier's current moving environment;

[0134] S402: inputting the geomagnetic information of the area to be track planned and the environmental information of the movement environment of the target carrier at the current moment into the track planning model, so as to determine the movement decision information of the target carrier at the next moment through the track planning model, wherein the track planning model is trained by the above-mentioned track planning model training method;

[0135] S403: Performing track planning for the target carrier according to the motion decision information of the target carrier at each moment.

[0136] In practical applications, the server can obtain the geomagnetic information of the area to be tracked and the environmental information collected by the target carrier through sensors, and input it into the track planning model, so as to determine the motion decision information of the target carrier at each moment through the decision network in the track planning model. Based on these motion decision information, the motion direction of the target carrier at each moment can be further determined, thereby further generating the motion trajectory of the target carrier in the area to be tracked.

[0137] During the movement of the target carrier, real-time data can be collected by vector geomagnetic sensors, radars, cameras and other sensors mounted on the carrier, and after signal processing, it can be used to correct the current geomagnetic information and environmental information of the detection grid and update them synchronously.

[0138] Furthermore, in order to ensure the accuracy of geomagnetic navigation, the server can determine the geomagnetic adaptation area in the area to be planned according to the geomagnetic information and environmental information of the area to be planned, and each geomagnetic adaptation search area is set to include m×n grid areas, where 0<m<M, 0<n<N. In practical applications, the geomagnetic adaptation area can be a geomagnetic field area with a good geomagnetic adaptation rate in the target area. These areas usually have obvious characteristics, rich information, and strong adaptability, which helps to improve the accuracy and stability of geomagnetic matching.

[0139] After determining the motion decision information of the target carrier in the area to be tracked, the server can further perform local dynamic optimization of the motion trajectory of the target carrier in the area to be tracked, with the constraint that each track point of the target carrier during the movement is located in the geomagnetic adaptation area, thereby generating a real-time trajectory of the target movement, and then performing geomagnetic navigation on the target carrier based on the target motion trajectory.

[0140] It should be noted that in the process of local dynamic optimization of the motion trajectory, the aforementioned deep reinforcement learning method can continue to be used, but the geomagnetic adaptation rate is not used to calculate the reward value. Of course, the server can also use other local trajectory planning methods such as D* (D-Star) algorithm and TEB (Time Elastic Band) algorithm.

[0141] For ease of understanding, this specification provides a schematic diagram of the motion trajectory based on the geomagnetic adaptation area, such as Figure 5 shown.

[0142] Figure 5 A schematic diagram of a motion trajectory based on the geomagnetic adaptation zone provided in this specification.

[0143] Among them, the grid area represents the geomagnetic adaptation area, the black filled area represents the obstacle threat area, and the black dotted line represents the movement trajectory of the target carrier.

[0144] Of course, during model training and actual geomagnetic navigation, the geomagnetic adaptation zone can also be determined through the decision network, and the motion decision information can be determined under the constraints of the geomagnetic adaptation zone. In this way, since the process of determining the motion decision information is subject to the constraints of the geomagnetic adaptation zone, the target carrier's track points at each moment will be located within the geomagnetic adaptation zone. In this case, there is no need to adjust the motion trajectory.

[0145] It can be seen from the above method that this scheme realizes the geomagnetic adaptation zone search and global track synchronization planning by integrating the information of the carrier's geomagnetic field and the carrier's motion environment, and designs relevant reward functions, thereby improving the efficiency of geomagnetic adaptation zone search and track planning at the same time; the adoption of deep reinforcement learning method enhances the generalization of track planning; further integration of real-time dynamically collected sensor information, and fine local track planning in the geomagnetic adaptation zone improves the dynamics of track planning, thereby providing a good foundation and conditions for the realization of long-duration and high-precision geomagnetic matching navigation algorithm.

[0146] The above are one or more implementations of the model training method and the geomagnetic navigation track planning method of this specification. Based on the same idea, this specification also provides a corresponding model training device and a geomagnetic navigation track planning device, such as Figure 6 , Figure 7shown.

[0147] Figure 6 A schematic diagram of a model training device provided in this specification includes:

[0148] An acquisition module 601 is used to acquire geomagnetic information of a target area and environmental information of a moving environment where a target carrier is located at a first moment, as state information at the first moment;

[0149] A decision module 602, used for inputting the state information at the first moment into the trajectory planning model to be trained, so as to determine the motion decision information of the target carrier through the decision network in the trajectory planning model;

[0150] A determination module 603 is used to determine the state information at a second moment according to the motion decision information and the state information at the first moment, wherein the second moment is a moment next to the first moment;

[0151] A reward module 604, configured to determine a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment;

[0152] A value module 605, configured to input the state information at the first moment and the state information at the second moment into a value network of the trajectory planning model, so as to determine, through the value network, an evaluation value corresponding to the state information at the first moment and an evaluation value corresponding to the state information at the second moment;

[0153] The training module 606 is used to determine the loss value of the trajectory planning model according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment and the reward value, and train the trajectory planning model according to the loss value.

[0154] Optionally, the environmental information includes at least: location information of the target carrier at the first moment, location information of the target point, obstacle and threat area information, explored area information and unexplored area information.

[0155] Optionally, the reward module 604 is specifically used to determine the geomagnetic adaptation rate corresponding to the motion trajectory of the target carrier from the first moment to the second moment based on the status information at the first moment and the status information at the second moment; if the geomagnetic adaptation rate is higher than the average geomagnetic adaptation rate of the target area, increase the reward value, otherwise reduce the reward value.

[0156] Optionally, the reward module 604 is specifically used to determine the distance of the target carrier relative to the target point at the first moment based on the state information at the first moment, and to determine the distance of the target carrier relative to the target point at the second moment based on the state information at the second moment; if the distance of the target carrier relative to the target point at the second moment is less than the distance of the target carrier relative to the target point at the first moment, increase the reward value, otherwise reduce the reward value.

[0157] Optionally, the reward module 604 is specifically used to determine, based on the status information at the second moment, the distance of the target carrier relative to the center of the influence area and the center of the threat area of ​​the obstacle at the second moment as a discrimination distance; based on the discrimination distance, determine whether the target carrier is located in the obstacle influence area and / or threat area at the second moment; if so, reduce the reward value.

[0158] Optionally, the reward module 604 is specifically used to determine whether the target carrier can reach the target point within the maximum range allowed by the remaining fuel of the target carrier based on the status information at the first moment and the status information at the second moment; if the target carrier can reach the target point within the maximum range allowed by the remaining fuel, increase the reward value, otherwise reduce the reward value.

[0159] Optionally, the reward module 604 is specifically used to determine the expected evaluation value based on the evaluation value corresponding to the state information at the second moment and the reward value; and determine the loss value based on the deviation between the evaluation value corresponding to the state information at the first moment and the expected evaluation value.

[0160] Optionally, the training module 606 is specifically used to train the value network of the trajectory planning model with minimizing the loss value as the optimization goal, and to train the decision network of the trajectory planning model with maximizing the expected evaluation value as the optimization goal.

[0161] Figure 7 A schematic diagram of a geomagnetic navigation track planning device provided in this specification includes:

[0162] The acquisition module 701 is used to acquire the geomagnetic information updated at the current moment in the area to be planned and the environmental information of the movement environment where the target carrier is located at the current moment;

[0163] An input module 702 is used to input the geomagnetic information of the area to be track planned at the current moment and the environmental information of the movement environment of the target carrier at the current moment into the track planning model, so as to determine the movement decision information of the target carrier at the next moment through the track planning model;

[0164] The planning module 703 is used to plan the trajectory of the target carrier according to the motion decision information of the target carrier at each moment.

[0165] Optionally, before performing geomagnetic navigation on the target carrier according to the motion decision information of the target carrier at each moment, the input module 702 is further used to determine the geomagnetic adaptation area in the area to be track planned according to the geomagnetic information of the area to be track planned and the environmental information;

[0166] The planning module 703 is specifically used to perform local track planning for the target carrier according to the motion decision information of the target carrier at each moment, with the constraint that each track point of the target carrier during the motion is located in the geomagnetic adaptation area.

[0167] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A model training method and Figure 4 A trajectory planning method for geomagnetic navigation is provided.

[0168] This manual also provides Figure 8 The one shown corresponds to Figure 1 or Figure 4 A schematic diagram of the electronic device. Figure 8 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A model training method and Figure 4 Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0169] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0170] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0171] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0172] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0173] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0177] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0178] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0179] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0180] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0181] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0183] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0184] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A model training method, characterized in that: include: Acquire geomagnetic information of the target area and environmental information of the moving environment of the target carrier at the first moment as the state information at the first moment; Inputting the state information at the first moment into the trajectory planning model to be trained, so as to determine the motion decision information of the target carrier through the decision network in the trajectory planning model; Determine the state information at a second moment according to the motion decision information and the state information at the first moment, wherein the second moment is a moment after the first moment; determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment; Inputting the state information at the first moment and the state information at the second moment into the value network of the trajectory planning model, so as to determine the evaluation value corresponding to the state information at the first moment and the evaluation value corresponding to the state information at the second moment through the value network; The loss value of the trajectory planning model is determined according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment, and the reward value, and the trajectory planning model is trained according to the loss value.

2. The method according to claim 1, characterized in that The environmental information at least includes: the position information of the target carrier at the first moment, the position information of the target point, obstacle and threat area information, explored area information and unexplored area information.

3. The method according to claim 1, characterized in that Determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes: Determining, according to the state information at the first moment and the state information at the second moment, a geomagnetic adaptation rate corresponding to the motion trajectory of the target carrier from the first moment to the second moment; If the geomagnetic matching rate is higher than the average geomagnetic matching rate of the target area, the reward value is increased; otherwise, the reward value is reduced.

4. The method according to claim 1, characterized in that Determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes: Determine the distance of the target carrier relative to the target point at the first moment according to the state information at the first moment, and determine the distance of the target carrier relative to the target point at the second moment according to the state information at the second moment; If the distance of the target carrier relative to the target point at the second moment is smaller than the distance of the target carrier relative to the target point at the first moment, the reward value is increased; otherwise, the reward value is decreased.

5. The method according to claim 1, characterized in that Determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes: Determine, according to the state information at the second moment, the distance of the target carrier relative to the center of the influence area and the center of the threat area of ​​the obstacle at the second moment as a determination distance; Determining, according to the determination distance, whether the target carrier is located in the influence area and / or threat area of ​​the obstacle at the second moment; If so, the reward value is decreased.

6. The method according to claim 1, characterized in that Determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment specifically includes: According to the state information at the first moment and the state information at the second moment, determining whether the target carrier can reach the target point within the maximum range allowed by the remaining fuel of the target carrier; If the target carrier can reach the target point within the maximum range allowed by the remaining fuel, the reward value is increased; otherwise, the reward value is reduced.

7. The method according to claim 1, characterized in that Determining the loss value of the trajectory planning model according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment, and the reward value specifically includes: determining an expected evaluation value according to the evaluation value corresponding to the state information at the second moment and the reward value; The loss value is determined according to a deviation between the evaluation value corresponding to the state information at the first moment and the expected evaluation value.

8. The method according to claim 7, characterized in that The trajectory planning model is trained according to the loss value, specifically including: Taking minimizing the loss value as the optimization goal, the value network of the trajectory planning model is trained, and taking maximizing the expected evaluation value as the optimization goal, the decision network of the trajectory planning model is trained.

9. A geomagnetic navigation track planning method, characterized in that: include: Obtain the geomagnetic information updated at the current moment in the area to be planned and the environmental information of the target carrier's motion environment at the current moment; Inputting the geomagnetic information updated at the current moment in the area to be track planned and the environmental information of the movement environment in which the target carrier is located at the current moment into the track planning model, so as to determine the movement decision information of the target carrier at the next moment through the track planning model, wherein the track planning model is trained by the method described in any one of claims 1 to 8 above; According to the motion decision information of the target carrier at each moment, the trajectory planning is performed for the target carrier.

10. The method according to claim 9, characterized in that Before performing track planning for the target carrier according to the motion decision information of the target carrier at each moment, the method further includes: Determine a geomagnetic adaptation area in the area to be planned according to the geomagnetic information of the area to be planned and the environmental information; According to the motion decision information of the target carrier at each moment, the trajectory planning of the target carrier is carried out, specifically including: With the constraint that each track point of the target carrier during its movement is located in the geomagnetic adaptation area, local track planning is performed on the target carrier according to the movement decision information of the target carrier at each moment.

11. A model training device, characterized in that: include: An acquisition module, used to acquire geomagnetic information of a target area and environmental information of a moving environment where a target carrier is located at a first moment, as state information at the first moment; A decision module, used for inputting the state information at the first moment into the trajectory planning model to be trained, so as to determine the motion decision information of the target carrier through the decision network in the trajectory planning model; a determination module, configured to determine state information at a second moment according to the motion decision information and the state information at the first moment, wherein the second moment is a moment next to the first moment; A reward module, used for determining a reward value for the motion decision information according to the state information at the first moment and the state information at the second moment; a value module, used for inputting the state information at the first moment and the state information at the second moment into the value network of the trajectory planning model, so as to determine the evaluation value corresponding to the state information at the first moment and the evaluation value corresponding to the state information at the second moment through the value network; A training module is used to determine the loss value of the trajectory planning model according to the evaluation value of the state information at the first moment, the evaluation value corresponding to the state information at the second moment and the reward value, and to train the trajectory planning model according to the loss value.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 10 is implemented.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

Images