Geomagnetic vector matching navigation method based on deep reinforcement learning and related device

Abstract

The invention provides a geomagnetic vector matching navigation method based on deep reinforcement learning and a related device. The electronic equipment obtains a plurality of track points, and determines a plurality of reference track points corresponding to the plurality of track points from a standard geomagnetic isoline according to the actually measured geomagnetic information of the plurality of track points; obtaining a rotation deviation and a translation deviation according to the plurality of track points and the plurality of reference track points; and adjusting the positions of the plurality of track points according to the rotation deviation and the translation deviation to obtain a plurality of updated track points. Therefore, a plurality of pieces of actually measured geomagnetic information on the standard geomagnetic isoline are introduced to serve as the reference basis, it is ensured that the error correction process has clear physical significance and reliable reference basis, and the accuracy of error correction is improved by correcting the translation error and the rotation error at the same time. The problem of insufficient positioning precision caused by single-dimension correction in a traditional geomagnetic contour matching method is solved, so that the positioning precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of navigation, in particular to a geomagnetic vector matching navigation method based on deep reinforcement learning and a related device. BACKGROUND

[0002] As a kind of autonomous navigation technology, geomagnetic matching navigation has significant advantages in satellite signal shielding and visual feature unobvious scenes due to its characteristic of not needing to rely on external signal source.However, the current navigation method based on geomagnetic profile matching realizes positioning function by making the geomagnetic field of matching position and geomagnetic measurement value features as close as possible.But in error correction aspect, geomagnetic profile matching algorithm only corrects translation error, and ignores the potential influence of rotation error on overall positioning effect, resulting in insufficient positioning accuracy of algorithm. SUMMARY

[0003] In order to overcome at least one of the deficiencies in the prior art, the present application provides a geomagnetic vector matching navigation method based on deep reinforcement learning and a related device, which uses multiple measured geomagnetic information as evaluation criteria for correction on the basis of standard geomagnetic contour line, ensures that the error correction process has clear physical meaning and reliable reference basis, and solves the problem of insufficient positioning accuracy caused by single-dimensional correction in traditional geomagnetic profile matching method by simultaneously correcting translation error and rotation error, thereby improving positioning accuracy.

[0004] In a first aspect, the present application provides a geomagnetic vector matching navigation method based on deep reinforcement learning, which comprises: acquiring multiple track points; determining multiple reference track points corresponding to the multiple track points from standard geomagnetic contour line according to measured geomagnetic information of the multiple track points, wherein the distance between each reference track point and corresponding track point satisfies a preset constraint condition; obtaining rotation deviation and translation deviation according to the multiple track points and the multiple reference track points; adjusting the position of the multiple track points according to the rotation deviation and the translation deviation to obtain updated multiple track points.

[0005] In a second aspect, the present application provides a geomagnetic vector matching navigation device based on deep reinforcement learning, which comprises: a data acquisition module for acquiring multiple track points; a data correction module for determining multiple reference track points corresponding to the multiple track points from standard geomagnetic contour line according to measured geomagnetic information of the multiple track points, wherein the distance between each reference track point and corresponding track point satisfies a preset constraint condition; The data correction module is further configured to obtain a rotation deviation and a translation deviation based on the plurality of track points and the plurality of reference track points; The data correction module is further configured to adjust the positions of the plurality of track points according to the rotation deviation and the translation deviation to obtain a plurality of updated track points.

[0006] In a third aspect, the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the geomagnetic vector matching navigation method based on deep reinforcement learning.

[0007] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements the geomagnetic vector matching navigation method based on deep reinforcement learning.

[0008] Compared with the prior art, this application has the following beneficial effects: The present application provides a geomagnetic vector matching navigation method and related devices based on deep reinforcement learning. The electronic device obtains multiple track points, and according to the measured geomagnetic information of the multiple track points, determines multiple reference track points corresponding to the multiple track points from the standard geomagnetic contour line; wherein the distance between each measured geomagnetic information and the corresponding track point satisfies the preset constraint condition; according to the multiple track points and the multiple reference track points, the rotation deviation and the translation deviation are obtained; according to the rotation deviation and the translation deviation, the positions of the multiple track points are adjusted to obtain the updated multiple track points. In this way, the multiple measured geomagnetic information on the standard geomagnetic contour line is introduced as a reference benchmark, ensuring that the error correction process has a clear physical meaning and a reliable reference basis, and by simultaneously correcting the translation error and the rotation error, the problem of insufficient positioning accuracy caused by single-dimensional correction in the traditional method is solved, thereby improving the positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 One of the flow charts of the geomagnetic vector matching navigation method based on deep reinforcement learning provided in an embodiment of the present application; Figure 2 A schematic diagram of sliding a sliding window provided in an embodiment of the present application; Figure 3 The second flowchart of the geomagnetic vector matching navigation method based on deep reinforcement learning provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a geomagnetic vector matching navigation device based on deep reinforcement learning provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present application (hereinafter referred to as the embodiments) more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0012] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0013] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0014] In the description of this application, it should be noted that the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be understood as indicating or implying relative importance. In addition, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0015] Based on the above statement, as introduced in the background technology, the geomagnetic contour matching algorithm only corrects the translation error, but ignores the potential impact of the rotation error on the overall positioning effect, resulting in insufficient positioning accuracy of the algorithm.

[0016] Based on the discovery of the above technical problems, the following technical solutions are proposed after creative work to solve or improve the above problems. It should be noted that the defects existing in the solutions in the above prior art are the results obtained after practice and careful study. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of this application for the above problems below should be regarded as contributions to this application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0017] In view of this, this embodiment provides a geomagnetic vector matching navigation method based on deep reinforcement learning. Figure 1 As shown, the method includes: S1, obtain multiple track points.

[0018] S2, determining a plurality of reference track points corresponding to the plurality of track points from a standard geomagnetic contour line according to the measured geomagnetic information of the plurality of track points.

[0019] The distance between each measured geomagnetic information and the corresponding track point meets the preset constraint conditions.

[0020] S3: Obtain rotational deviation and translational deviation according to the multiple track points and the multiple reference track points.

[0021] S4, adjusting the positions of the plurality of track points according to the rotation deviation and the translation deviation to obtain a plurality of updated track points.

[0022] In this way, multiple measured geomagnetic information is used as evaluation criteria for correction based on standard geomagnetic contour lines, ensuring that the error correction process has clear physical meaning and reliable reference basis. By simultaneously correcting translation errors and rotation errors, the insufficient positioning accuracy caused by single-dimensional correction in traditional geomagnetic contour matching methods is solved, thereby improving positioning accuracy.

[0023] It should be understood that for the geomagnetic vector matching navigation method based on deep reinforcement learning provided in this embodiment, the electronic device implementing the method may be, but is not limited to, a mobile terminal, a tablet computer, a laptop computer, a server, a mobile vehicle, etc. The mobile vehicle may be, but is not limited to, a drone, an autonomous vehicle, an unmanned ground vehicle, an unmanned underwater vehicle, etc.

[0024] To make the solution provided by this embodiment clearer, the following uses a drone as an electronic device to implement the method, and describes in detail the various steps of the method shown in the figure. However, it should be understood that the operations in the flowchart can be implemented in a non-sequential order, and steps that have no logical contextual relationship can be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart, or remove one or more operations from the flowchart, guided by the content of this application. Figure 1 , the method comprising: S1, obtain multiple track points.

[0025] During the execution of the above steps, the drone uses multiple navigation methods simultaneously. For example, these include an inertial navigation system and a geomagnetic navigation system. During flight, the drone collects navigation data generated by the inertial navigation system at a fixed frequency, and simultaneously obtains measured geomagnetic information measured by the geomagnetic navigation equipment. The inertial navigation system generates a series of coordinate points through built-in sensors. Each coordinate point represents a track point during the drone's flight. These track points are marked on the map as the drone's movement trajectory within a certain period of time. At the same time, the measured geomagnetic information synchronously recorded by the geomagnetic navigation equipment includes information on the strength and direction of the geomagnetic field. In actual applications, since the external environment may interfere with the measured geomagnetic information, the collected measured geomagnetic information must also be subjected to magnetic interference compensation processing to ensure its accuracy and reliability.

[0026] like Figure 2 As shown, in this embodiment, in order to improve the correction efficiency of the track points, the UAV does not use all the track points for correction at the same time, but selectively intercepts a part of the track points as the analysis object. Specifically, the size of the sliding window is set to , which means intercepting continuous track points (the sliding window in the figure intercepts 5 track points). Therefore, the drone only performs the correction operation based on geomagnetic matching on the 5 track points within the coverage of the sliding window each time. After the correction is completed, the last track point in the sliding window will be output. The updated position of the last track point (i.e., the last track point at the end) is considered the correction result at the current moment. This updated last track point becomes the new correction track point.

[0027] It's important to note that the multiple trackpoints within the sliding window can be divided into two categories: corrected trackpoints and recently added trackpoints. Corrected trackpoints are those that have already been corrected, while recently added trackpoints are located at the end of the sliding window and represent those that require focused correction. As the drone continues to move, the sliding window will continue to slide forward, incorporating the latest trackpoints and removing the oldest trackpoints, ensuring that the correction process remains focused on the current flight status.

[0028] S2, determining a plurality of reference track points corresponding to the plurality of track points from the standard geomagnetic contour according to the measured geomagnetic information of the plurality of track points.

[0029] It should be understood that the standard geomagnetic contour refers to a set of contour data constructed in advance, reflecting the distribution characteristics of the geomagnetic field in a certain region. Similar to the contour line, each contour line connects points with the same geomagnetic field strength or direction characteristics, forming a complete geomagnetic field distribution map, so that the position of any point on the geomagnetic contour has the same geomagnetic field. These contours are generated based on high-precision geomagnetic measurements or simulations, and can accurately describe the geomagnetic field strength and direction information at different positions in the region. The geomagnetic contour can be represented in two or three dimensions, and usually includes the contour distribution of multiple components of the geomagnetic field (such as X component, Y component and Z component).

[0030] It should also be understood that the distance between each measured geomagnetic information and the corresponding track point satisfies a preset constraint condition. The preset constraint condition can be the nearest point on the geomagnetic contour to the track point. Of course, it can also be any point within a preset range on the geomagnetic contour from the track point. In addition, it should also be understood that the geomagnetic contour is generated from a standard geomagnetic vector reference map constructed in advance, which reflects the actual spatial distribution characteristics of the magnetic field three components (i.e. X, Y, Z three direction components) in the navigation region, so it means that there are geomagnetic contours on X, Y, Z three direction components.

[0031] Taking the nearest point as an example, the unmanned aerial vehicle determines the reference track point according to the track point provided by the inertial navigation system and the real-time collected measured geomagnetic information; matches the measured geomagnetic information with the contour in the standard geomagnetic vector reference map to determine the geomagnetic contour corresponding to the measured geomagnetic information, and determines the nearest point to the track point on the geomagnetic contour as the reference track point.

[0032] Here, the reference track point is represented as , the nearest neighbor points on the geomagnetic value contours of the three components are represented as , , , the final reference track point can be calculated by the following formula :

[0033] wherein, , , are the weighting coefficients of the three components, used to adjust the importance of different direction components. Therefore, this weighting process can consider the distribution characteristics of each component of the magnetic field, so as to more accurately determine the position of the reference track point.

[0034] In this embodiment, for A sliding window is used, and the objective function of the optimization is set as:

[0035] Therefore, the update method of the track point can be expressed as:

[0036] in, is the learning rate, yes right The gradient, It is The first Track points, is with The corresponding reference track point.

[0037] Based on the description of the reference track points in the above embodiment, the following Figure 1 Step S3 in the following is explained: S3: Obtain rotational deviation and translational deviation according to the multiple track points and the multiple reference track points.

[0038] As an optional implementation, step S3 may include: S3-1, according to each track point and the corresponding reference track point, obtain the angle between the two points.

[0039] S3-2, obtaining the rotation deviation based on the statistical characteristics of multiple angles.

[0040] During the execution of the above steps, the drone expresses the track point and the reference track point in vector form and calculates the angular relationship between the two using the angle formula between the two vectors. Therefore, a series of angle values ​​can be obtained, which reflect the directional differences between the track point and the corresponding reference track point. In order to integrate the error conditions of multiple track points, the drone can use the second update rate to adjust the mean between multiple angles to obtain a rotation deviation. In this regard, it can be understood that all angle values ​​are summed and divided by the total number of angles to obtain a rotation deviation in the form of an expected value. The rotation deviation is used to describe the overall rotation error of the track point relative to the reference track point within the entire sliding window.

[0041] For example, for A sliding window, where the track points are represented as , and the corresponding reference track point is expressed as , so the rotation deviation The expression is:

[0042]

[0043] wherein, denotes the average of the plurality of included angles, is the rotation angle deviation of the second update rate, denotes the i-th iteration.It can be understood that, for the track points in the i-th sliding window, the number of times of correction is not limited to one, but can be multiple times. Based on the above description of the rotation deviation in the embodiment, step S3 further comprises: S3-3, obtaining the translation deviation according to the rotation deviation.

[0044] For the calculation of the translation deviation, the present embodiment combines the rotation angle deviation in the above embodiment. Therefore, as an optional implementation manner, the unmanned aerial vehicle can obtain a rotation matrix according to the rotation deviation, wherein the rotation matrix takes the rotation deviation as a rotation angle; apply the rotation matrix to the plurality of track points to obtain adjusted plurality of track points; compare the adjusted plurality of track points with the plurality of reference track points to obtain an expected deviation between the plurality of track points and the plurality of reference track points; adjust the expected deviation by using a first update rate to obtain the translation deviation.

[0045] Exemplarily, also for the i-th sliding window, the calculation manner of the translation deviation is as follows:

[0046]

[0047] wherein, denotes the rotation matrix of the i-th iteration, denotes the rotation deviation.

[0048]

[0049] wherein, denotes the expected deviation.

[0050]

[0051] wherein, is the first update rate of the translation deviation .

[0052] Based on the above description of the rotation deviation and the translation deviation in the embodiment, the step S4 in the above embodiment will be described as follows: Figure 1 ​​​​S4, adjusting the positions of the plurality of track points according to the rotation deviation and the translation deviation to obtain a plurality of updated track points.

[0053] In this embodiment, the above-mentioned rotation deviation and translation deviation are taken into consideration at the same time. Therefore, as an optional implementation of step S4, for each track point, the drone can use the rotation matrix based on the rotation deviation to act on the track point to obtain the rotation angle adjustment amount; and use the rotation angle adjustment amount and the translation deviation to correct the track point to obtain an updated track point.

[0054] For example, the track points of the sliding window can be iteratively corrected according to the following formula to obtain Track points after update :

[0055] Where, Indicates the It can be understood that if the track points are updated continuously, the The track point updated is .

[0056] In practice, it was found that although the insufficient positioning accuracy caused by single-dimensional correction in the traditional geomagnetic contour matching method can be solved by simultaneously correcting the translation error and the rotation error, the correction effect depends on the actual measurement accuracy of the geomagnetic field and the corresponding geomagnetic contour distribution accuracy. Therefore, it is not robust to disturbances and is prone to fall into local optimality, resulting in a decrease in positioning accuracy. In view of this, after the correction of the rotation deviation and the translation deviation, a comparative analysis can be performed with the standard geomagnetic information and the measured geomagnetic information to evaluate the quality of the current correction result and provide a basis for subsequent correction strategies. In this way, after a certain number of iterations, a more accurate positioning result can be obtained. To this end, this embodiment needs to consider three factors, such as Figure 3 As shown, the method further includes: S5, obtaining updated multiple standard geomagnetic information of the locations of the multiple track points and multiple measured geomagnetic information of the locations of the multiple track points.

[0057] S6, calculating first difference information between the plurality of standard geomagnetic information and the plurality of measured geomagnetic information.

[0058] During the above steps, the standard geomagnetic information corresponding to all track points within the current sliding window must first be read from the geomagnetic vector reference map. This standard geomagnetic information, derived from a pre-constructed geomagnetic vector reference map, reflects the geomagnetic field strength and direction characteristics at each location within the region. At the same time, actual measured geomagnetic information for the same set of track points must also be obtained. This measured geomagnetic information is collected in real time by the device during motion and, after necessary magnetic interference compensation, is used to characterize the actual geomagnetic field characteristics at the current location.

[0059] In order to quantify the degree of difference between the two sets of information, this embodiment introduces the mean square error. That is, by performing a square operation on the difference between the standard geomagnetic information and the measured geomagnetic information of each track point component by component and averaging the results, the mean square error of the geomagnetic vectors in all directions of the track sequence within the sliding window is obtained as the first difference information.

[0060] For example, the geomagnetic vector value of the current sliding window correction track sequence is read from the geomagnetic vector reference map , and calculate the corresponding geomagnetic vector measurement value The mean square error .

[0061]

[0062] Where, 、 、 Respectively represent The components of the standard geomagnetic information of the updated track point in the X, Y, and Z directions. Similarly, 、 、 Indicates the The components of the measured geomagnetic information at the location of each track point in the X, Y, and Z directions.

[0063] S7, obtaining the single standard geomagnetic information of the location of the last track point among the updated multiple track points and the single measured geomagnetic information of the location of the last track point among the multiple track points.

[0064] S8, calculating second difference information between the single standard geomagnetic information and the single measured geomagnetic information.

[0065] In this implementation, the standard and measured geomagnetic information corresponding to the last track point in the sliding window must be separately obtained. This allows for accurate matching of the current position, providing a more specific reference for subsequent navigation strategies. During these steps, the drone reads the standard and measured geomagnetic information for the last track point before and after the update from the geomagnetic vector reference map. The difference between the two is then used to directly calculate the geomagnetic vector error, which serves as the second difference between the two.

[0066] For example, let us continue to assume that there are trajectory points, so the expression of the second difference information is:

[0067] Where, represents the second difference information, Represents a single standard geomagnetic information, Represents a single measured geomagnetic information.

[0068] S9 , determining a subsequent correction strategy for the updated plurality of track points according to the last track point among the updated plurality of track points, the first difference information, and the second difference information.

[0069] To address this issue, this embodiment introduces an update strategy model to determine whether to continue correcting track points. The drone can input the last track point, the first difference information, and the second difference information from the updated track points into a pre-trained update strategy model to determine the subsequent correction strategy for the updated track points.

[0070] It can be understood that the update strategy model is obtained through deep reinforcement learning technology. The update strategy model can use the last state feature vector and related difference information of the updated multiple track points to generate subsequent correction strategies.

[0071] For example, considering that the update strategy model is a neural network-based model, it is necessary to construct a state feature vector for the current iteration cycle for input into the update strategy model. This state feature vector consists of the updated last track point in the sliding window, its corresponding geomagnetic vector error, and the mean square error of the geomagnetic vectors of multiple track points in the sliding window in each direction. These data reflect the current estimation accuracy, local geomagnetic matching deviation, and global geomagnetic matching deviation of the drone, respectively. The drone integrates the above three state component information into a state feature vector and inputs it into the update strategy model, which then predicts the next correction strategy.

[0072] In this embodiment, the update strategy model is trained by using a Q network. It should be understood that the Q network is a neural network structure based on deep reinforcement learning, which includes an input layer, two hidden layers and an output layer. The number of neurons in the input layer is consistent with the dimension of the state feature vector, ensuring that all information in the state feature vector can be completely received; each hidden layer is a fully connected layer, and the ReLU function is used as the activation function, which is used to extract the nonlinear relationship of the state features; the output layer is also a fully connected layer, and the number of output neurons is consistent with the dimension of the action space, corresponding to the possible action selection. For example, action 1 represents continuing to correct in the direction of the current iteration gradient; action 0 represents terminating the correction; and action -1 represents backtracking to the last estimated position and terminating the correction. It can be understood that as long as the update strategy model outputs action 1, the track points in the sliding window are continuously corrected until the maximum number of iterations is reached.

[0073] Thus, after at least one correction of the plurality of track points in the current sliding window, the corrected position of the last track point in the sliding window is retained. If the UAV is still flying, the track point at the next time is estimated according to the inertial navigation sensor data and at least part of the corrected track points in the current sliding window, then the sliding window is moved to add the track point at the next time to the sliding window (the corrected track point at the head of the sliding window is removed), and finally the track points in the current sliding window are continuously corrected.

[0074] For the training process of the update strategy model described above, the Q network can use an ε-greedy strategy as the action selection strategy. The strategy randomly selects an action with a probability of and selects the action with the maximum Q value estimated by the current Q network with a probability of After selecting an action, the current state, the selected action and the corresponding Q value estimate need to be recorded, and the state feature vector at the next time is calculated according to the selected action. Then the state feature vector is input into the Q network again to obtain the Q value estimation vector at the next time, and the maximum value is selected as the optimal Q value estimate of the next state.

[0075] Further, in combination with the reward signal of the current action, the target value of the Q value of the current state is calculated through the Bellman equation of reinforcement learning. The design of the reward function is based on the distance change between the estimated position of the last track point and the true position of the last track point: when the estimated distance is shortened compared with the previous state, a positive reward of +1 is given; if the distance increases, a penalty of -1 is applied; if the distance remains unchanged, the reward is 0. In this way, the model is guided to gradually optimize the navigation strategy and improve the matching accuracy.

[0076] Based on the above description, the loss function of the Q network can be selected as the mean square error function, and the parameters of the Q network are updated and optimized through the back propagation algorithm. The corresponding expression is as follows:

[0077] Where, is the reward value of the current action, is the discount factor, between 0 and 1, Is the next state All possible moves The maximum Q value estimate in .

[0078]

[0079] Where, Is the current state The Q value estimate of is the target Q value, is the batch size.

[0080] It should be noted that in practical applications, in addition to using the Q-network for training, it is also possible to introduce an improved Q-network architecture or other high-performance deep reinforcement learning algorithms (such as the PPO algorithm) to further enhance model performance. This shows that the above method can not only effectively meet the needs of navigation in complex environments, but also continuously improve the robustness and adaptability of the algorithm through continuous learning.

[0081] Furthermore, in practice, we've discovered that when performing geomagnetic vector matching navigation, there can be significant initial deviations between the estimated position and the actual position. This deviation can be caused by a variety of factors, such as large cumulative errors in the inertial navigation system or a complex geomagnetic environment. To ensure effective convergence in subsequent fine-tuning steps, a preliminary correction of the initial track estimate is required during the initial matching.

[0082] Specifically, within a preset distance at the beginning of flight, the drone calculates the weighted mean square error (MSE) between the initial standard geomagnetic information (i.e., the theoretical geomagnetic vector value read from the standard geomagnetic reference map) corresponding to the initial track points within the current sliding window and their corresponding initial measured geomagnetic information. It should be understood that the weighted MSE is a comprehensive evaluation indicator that reflects the potential deviation between the current initial track points and their true positions. If the weighted MSE exceeds a pre-set threshold, it indicates that these initial track points may have deviated from a reasonable range and require further adjustment.

[0083] When the weighted mean square error exceeds a set threshold, a traditional geomagnetic matching method, such as magnetic contour matching (MAGCOM), can be used to coarsely locate the initial track point. It should be noted that while traditional geomagnetic matching methods have low accuracy, they can quickly determine a relatively reasonable area within the global scope, thereby updating the initial estimated position to this area.

[0084] For the geomagnetic vector matching navigation method based on deep reinforcement learning provided in this embodiment (referred to as geomagnetic navigation for short), a two-hour actual flight test was carried out together with satellite navigation and inertial navigation. It was found that the trajectory of geomagnetic navigation was relatively close to that of satellite navigation, with a root mean square error of 46m and a termination point matching error of 12.3m. The root mean square error of conventional inertial navigation was 4.587km, thus achieving good matching and positioning results.

[0085] Based on the same inventive concept as the geomagnetic vector matching navigation method based on deep reinforcement learning provided in this embodiment, this embodiment also provides a geomagnetic vector matching navigation device based on deep reinforcement learning, which includes at least one software function module that can be stored in a memory or solidified in an electronic device in the form of software. The processor in the electronic device is used to execute the executable module stored in the memory. For example, the software function module and computer program included in the device. Please refer to Figure 4 Functionally, the device can include: Data acquisition module, used to obtain multiple track points; A data correction module is used to determine a plurality of reference track points corresponding to the plurality of track points from a standard geomagnetic contour line based on measured geomagnetic information of the plurality of track points, wherein the distance between each reference track point and the corresponding track point satisfies a preset constraint condition; The data correction module is further used to obtain rotation deviation and translation deviation based on multiple track points and multiple reference track points; The data correction module is further used to adjust the positions of the multiple track points according to the rotation deviation and the translation deviation to obtain the updated multiple track points.

[0086] Optionally, the data correction module is further specifically configured to: According to each track point and the corresponding reference track point, the angle between the two points is obtained; According to the statistical characteristics of multiple angles, the rotation deviation is obtained; According to the rotation deviation, the translation deviation is obtained.

[0087] Optionally, the data correction module is further specifically configured to: According to the rotation deviation, a rotation matrix is ​​obtained, wherein the rotation matrix uses the rotation deviation as the rotation angle; Apply the rotation matrix to multiple track points to obtain multiple adjusted track points; Comparing the adjusted multiple track points with the multiple reference track points to obtain expected deviations between the multiple track points and the multiple reference track points; The expected deviation is adjusted using the first update rate to obtain a translational deviation.

[0088] Optionally, the data correction module is further specifically configured to: The second update rate is used to adjust the average of the multiple angles to obtain the rotation deviation.

[0089] Optionally, the data correction module is further specifically configured to: For each track point, the rotation matrix based on the rotation deviation is applied to the track point to obtain the rotation angle adjustment; The track points are corrected using the rotation angle adjustment amount and the translation deviation to obtain updated track points.

[0090] Optionally, the data correction module is further configured to: Obtaining updated multiple standard geomagnetic information of the locations of multiple track points and multiple measured geomagnetic information of the locations of multiple track points; Calculating first difference information between a plurality of standard geomagnetic information and a plurality of measured geomagnetic information; Obtaining updated single standard geomagnetic information of the location of the last track point among the multiple track points and single measured geomagnetic information of the location of the last track point among the multiple track points; Calculating second difference information between the single standard geomagnetic information and the single measured geomagnetic information; A subsequent correction strategy for the updated plurality of track points is determined according to the last track point in the updated plurality of track points, the first difference information, and the second difference information.

[0091] Optionally, the data correction module is further specifically configured to: The last track point, the first difference information, and the second difference information among the updated track points are input into a pre-trained update strategy model to obtain a subsequent correction strategy for the updated track points.

[0092] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0093] It should also be understood that if the above embodiments are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0094] Therefore, this embodiment also provides a computer-readable storage medium. The storage medium stores a computer program that, when executed by a processor, implements the geomagnetic vector matching navigation method based on deep reinforcement learning provided in this embodiment. The storage medium can be any medium capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0095] This embodiment provides an electronic device that implements a geomagnetic vector matching navigation method based on deep reinforcement learning. Figure 5 As shown, the electronic device may include a processor 22 and a memory 21. In addition, the memory 21 stores a computer program, and the processor implements the geomagnetic vector matching navigation method based on deep reinforcement learning provided in this embodiment by reading and executing the computer program corresponding to the above embodiment in the memory 21.

[0096] Continue to see Figure 5 The electronic device further includes a communication unit 23. The memory 21, the processor 22 and the communication unit 23 are electrically connected to each other directly or indirectly via a system bus 24 to achieve data transmission or interaction.

[0097] The memory 21 may be an information recording device based on any electronic, magnetic, optical or other physical principles, for recording execution instructions, data, etc. In some embodiments, the memory 21 may be, but is not limited to, a volatile memory, a non-volatile memory, a storage drive, etc.

[0098] In some embodiments, the volatile memory may be a random access memory (RAM); in some embodiments, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, etc.; in some embodiments, the storage drive may be a magnetic disk drive, a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or a similar storage medium, or a combination thereof.

[0099] The communication unit 23 is configured to transmit and receive data via a network. In some embodiments, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof. In some embodiments, the network may include one or more network access points. For example, the network may include a wired or wireless network access point, such as a base station and / or a network switching node, through which one or more components of the service request processing system may connect to the network to exchange data and / or information.

[0100] The processor 22 may be an integrated circuit chip having signal processing capabilities, and the processor may include one or more processing cores (e.g., a single-core processor or a multi-core processor). By way of example only, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction-set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), or a microprocessor, or any combination thereof.

[0101] I understand. Figure 5The structure shown is for reference only. Figure 5 More or fewer components than shown, or with Figure 5 Different configurations shown. Figure 5 The components shown may be implemented in hardware, software, or a combination thereof.

[0102] It should be understood that the devices and methods disclosed in the above embodiments may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, may be implemented using a dedicated hardware-based system that performs a specified function or action, or may be implemented using a combination of dedicated hardware and computer instructions.

[0103] The above descriptions are merely examples of various embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A geomagnetic vector matching navigation method based on deep reinforcement learning, characterized in that: The method comprises: Get multiple track points; Determining, based on the measured geomagnetic information of the plurality of track points, a plurality of reference track points corresponding to the plurality of track points from a standard geomagnetic contour line, wherein the distance between each of the reference track points and the corresponding track point satisfies a preset constraint condition; Obtaining a rotational deviation and a translational deviation according to the plurality of track points and the plurality of reference track points; The positions of the plurality of track points are adjusted according to the rotation deviation and the translation deviation to obtain a plurality of updated track points.

2. The geomagnetic vector matching navigation method based on deep reinforcement learning according to claim 1 is characterized in that: Obtaining a rotational deviation and a translational deviation according to the plurality of track points and the plurality of reference track points, including: Obtaining the angle between each track point and the corresponding reference track point; Obtaining the rotation deviation according to statistical characteristics of the plurality of angles; The translation deviation is obtained according to the rotation deviation.

3. The geomagnetic vector matching navigation method based on deep reinforcement learning according to claim 2 is characterized in that: Obtaining the translation deviation according to the rotation deviation includes: According to the rotation deviation, a rotation matrix is ​​obtained, wherein the rotation matrix uses the rotation deviation as a rotation angle; Applying the rotation matrix to the plurality of track points to obtain a plurality of adjusted track points; Comparing the adjusted track points with the reference track points to obtain expected deviations between the track points and the reference track points; The expected deviation is adjusted using a first update rate to obtain the translation deviation.

4. The geomagnetic vector matching navigation method based on deep reinforcement learning according to claim 2 is characterized in that: The rotation deviation is obtained according to the statistical characteristics of the plurality of angles, including: The average of the plurality of included angles is adjusted using a second update rate to obtain the rotation deviation.

5. The geomagnetic vector matching navigation method based on deep reinforcement learning according to claim 1, characterized in that: Adjusting the positions of the plurality of track points according to the rotational deviation and the translational deviation to obtain a plurality of updated track points includes: For each of the track points, a rotation matrix based on the rotation deviation is applied to the track point to obtain a rotation angle adjustment; The track point is corrected using the rotation angle adjustment amount and the translation deviation to obtain an updated track point.

6. The geomagnetic vector matching navigation method based on deep reinforcement learning according to claim 1, characterized in that: The method further comprises: Acquire a plurality of standard geomagnetic information of the locations of the updated plurality of track points and a plurality of measured geomagnetic information of the locations of the plurality of track points; Calculating first difference information between the plurality of standard geomagnetic information and the plurality of measured geomagnetic information; Acquire single standard geomagnetic information of the location of the last track point among the updated multiple track points and single measured geomagnetic information of the location of the last track point among the multiple track points; Calculating second difference information between the single standard geomagnetic information and the single measured geomagnetic information; A subsequent correction strategy for the updated plurality of track points is determined according to the last track point in the updated plurality of track points, the first difference information, and the second difference information.

7. The geomagnetic vector matching navigation method based on deep reinforcement learning according to claim 6, characterized in that: Determining a subsequent correction strategy for the updated multiple track points based on the last one of the updated multiple track points, the first difference information, and the second difference information, including: The last track point among the updated multiple track points, the first difference information, and the second difference information are input into a pre-trained update strategy model to obtain a subsequent correction strategy for the updated multiple track points.

8. A geomagnetic vector matching navigation device based on deep reinforcement learning, characterized in that: The device comprises: Data acquisition module, used to obtain multiple track points; a data correction module, configured to determine, based on the measured geomagnetic information of the plurality of track points, a plurality of reference track points corresponding to the plurality of track points from a standard geomagnetic contour line, wherein the distance between each of the reference track points and the corresponding track point satisfies a preset constraint condition; The data correction module is further configured to obtain a rotation deviation and a translation deviation based on the plurality of track points and the plurality of reference track points; The data correction module is further configured to adjust the positions of the plurality of track points according to the rotation deviation and the translation deviation to obtain a plurality of updated track points.

9. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the geomagnetic vector matching navigation method based on deep reinforcement learning as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, it implements the geomagnetic vector matching navigation method based on deep reinforcement learning as described in any one of claims 1 to 7.

Images