Large-scale aircraft positioning method and device based on satellite vector map

Abstract

The application provides a large-scale aircraft positioning method based on a satellite vector map, and relates to the technical field of unmanned aerial vehicle matching positioning, and the method comprises the following steps: acquiring an image of an aircraft to be positioned by using an overhead camera, inputting the image of the aircraft to be positioned into a semantic neural network, and obtaining a building semantic map; acquiring a satellite semantic vector map corresponding to the position of the image of the aircraft to be positioned, calculating the vector features of the building shapes in the satellite semantic vector map, and obtaining a reference map; performing shape matching on the building targets in the building semantic map based on the reference map, obtaining a coarse matching result, calculating the correlation values of the building targets in the building semantic map corresponding to the coarse matching result, screening the building targets according to the correlation values, and obtaining a fine matching result; and performing particle filtering on the fine matching result, and obtaining the global positioning of the aircraft to be positioned. The application realizes accurate positioning of the aircraft by using the above scheme.

Description

Technical Field

[0001] The present application relates to the field of UAV matching and positioning technology, and in particular to a large-scale aircraft positioning method and device based on satellite vector maps. Background Art

[0002] Satellite navigation positioning is the best global high-precision positioning method, but in some important missions, GPS alternative systems are also important. Visual matching positioning is one of the two methods based on vision positioning, which can find the current position of the aircraft in a reference satellite map.

[0003] Numerous research and methodologies have been proposed in this field. Existing technologies include template-based, feature-based, deep learning-based, and sequential navigation-based approaches. The primary goal of these technologies is to achieve higher matching accuracy within a small scale. However, large-scale matching and positioning are crucial for security missions where GPS is not an option. In some scenarios, such as deserts and forests, conventional matching and positioning are ineffective. In these situations, inertial navigation is the primary method for ensuring positioning accuracy. After flying over these areas, the cumulative error in inertial navigation can reach over ten kilometers.

[0004] Image matching only works when images are similar and is not suitable for situations where images differ significantly. This makes it inapplicable in practical positioning applications, especially in dense urban areas or with large-scale reference maps. Storing large reference maps is also cumbersome. Using high-resolution imagery takes up a lot of space and is slow, while using low-resolution images results in a higher probability of mismatches. Using semantic map matching can overcome the problem of image differences and reduce memory usage. Deep learning can segment different semantic objects at the pixel level and store the semantic segmentation information as a vector map, which is much smaller than an image map.

[0005] Buildings and roads are the most common semantic targets, so matching methods based on buildings and roads have been widely studied. Existing matching methods based on buildings and roads include: using building shape matching methods to reduce matching errors; positioning methods based on building ratio maps; calculating building ratios for matching; using structural information between adjacent buildings to establish a hash table for matching; using keypoint features for road matching; and using multi-task and multi-stage neural network methods for semantic segmentation and positioning.

[0006] All of the above matching methods require accurate semantic object recognition. In some cases, such as when adjacent buildings are identified as a single building, the narrow field of view can lead to limited and incomplete object recognition, making them less practical. Even with accurate semantic recognition, image matching can still be challenging, especially when the reference map is large. Semantic maps can contain very similar locations, making mismatches more likely. Summary of the Invention

[0007] The present application aims to solve one of the technical problems in the related art at least to a certain extent.

[0008] To this end, the first purpose of this application is to propose a large-scale aircraft positioning method based on satellite vector maps, which solves the technical problems that existing matching methods cannot guarantee accurate positioning and are prone to mismatching, and realizes accurate positioning of aircraft.

[0009] The second purpose of this application is to propose a large-scale aircraft positioning device based on satellite vector maps.

[0010] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a large-scale aircraft positioning method based on a satellite vector map, including: obtaining an image of the aircraft to be positioned through a bird's-eye view camera, and inputting the image of the aircraft to be positioned into a semantic neural network to obtain a building semantic map; obtaining a satellite semantic vector map of the corresponding position of the image of the aircraft to be positioned, and calculating the vector features of the building shape in the satellite semantic vector map to obtain a reference map; performing shape matching on the building targets in the building semantic map based on the reference map to obtain a coarse matching result, calculating the correlation value of the building targets in the building semantic map corresponding to the coarse matching result, and screening the building targets according to the correlation value to obtain a fine matching result; performing particle filtering on the fine matching result to obtain the global positioning of the aircraft to be positioned.

[0011] The large-scale aircraft positioning method based on satellite vector maps in the embodiment of the present application uses a downward-looking camera to capture aircraft images, uses satellite maps and captured images for matching and alignment, uses satellite semantic vector maps as matching maps, and combines NCC and shape information to match building semantic maps. After matching, all possible matching results are retained, and particle filtering is used to filter out the correct results, thereby achieving accurate positioning of the aircraft.

[0012] Optionally, in one embodiment of the present application, the vector feature of the building shape in the semantic vector map is a shape feature, and the shape feature is expressed as:

[0013]

[0014] Among them, f A Represents the shape feature vector of point A, s ij Indicates the building area of ​​the region;

[0015] Shape matching uses shape features for matching. The similarity between each two sets of shape features is calculated as follows:

[0016] dis({f A f B f C ...},{f A′f B′ f C′ ...})=min(dis(f S ,f S′ )),S∈{ABC}

[0017] Among them, f A represents the shape feature vector of point A, and S represents the point set {A, B, C}.

[0018] Optionally, in one embodiment of the present application, the calculation method of the relevant value of the building target is:

[0019]

[0020] Among them, V cam Represents the building outline of the building semantic graph, V ref Represents the building outline of the reference map, V box Represents the scope of the building semantic graph.

[0021] Optionally, in one embodiment of the present application, particle filtering is performed on the precise matching result to obtain the global positioning of the aircraft to be positioned, including:

[0022] Convert all building targets corresponding to the precise matching results in the building semantic graph into particles, where each particle represents a building target, and the attributes of each particle include the absolute position of the particle, the particle weight, and the number of times the particle is successfully matched;

[0023] Initialize all particles and obtain the initialization results;

[0024] Obtain inertial data through the inertial device, perform local mileage propagation on the initialization result based on the particle data and inertial data, and obtain a recursive result, where the recursive result represents the prior state estimate of the particle position at the current moment;

[0025] According to the recursive results, the particle data is measured, updated, and resampled to obtain updated particles;

[0026] The reliability of particles is evaluated according to the number of times each updated particle is successfully matched, and particles that pass the evaluation are defined as valid particles.

[0027] Optionally, in one embodiment of the present application, all particles are initialized, which is expressed as:

[0028]

[0029]

[0030] Where m represents the measurement value, f represents the filtered value, 0 represents time 0, θ represents the particle attribute, μ represents the position estimate, Σ represents the variance, α represents the filter weight, + represents the prior, i represents the i-th particle, and c represents the number of successful matches.

[0031] Optionally, in one embodiment of the present application, local mileage propagation is performed on the initialization result based on the aircraft image and inertial data, which is expressed as:

[0032]

[0033] Where t represents time t, f represents the filtered value, μ represents the position estimate, represents the odometer estimate of the relative displacement from time t-1 to time t, Represents variance.

[0034] Optionally, in one embodiment of the present application, the particle data is measured, updated, and resampled according to the recursive result, which is expressed as:

[0035]

[0036]

[0037] Among them, t represents time t, f represents the filter value, Σ represents the variance, + represents the prior, - represents the posterior, α represents the filter weight, and c represents the number of successful matches.

[0038] Optionally, in one embodiment of the present application, the reliability of the particles is evaluated based on the number of times each updated particle is successfully matched, which is expressed as:

[0039]

[0040]

[0041]

[0042] Among them, X t represents the absolute positioning result obtained by particle filtering, S1 and S2 are credibility indicators, m represents the measurement value, t represents time t, f represents the filtered value, c represents the number of successful matches, μ represents the position estimate, α represents the filter weight, and Σ represents the variance.

[0043] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a method including a target map acquisition module, a reference map acquisition module, a matching module, and a particle filtering module, wherein:

[0044] The module for acquiring the image of the aircraft to be located uses a bird's-eye view camera to obtain the image of the aircraft to be located, and inputs the image of the aircraft to be located into the semantic neural network to obtain a semantic image of the building;

[0045] A reference map acquisition module is used to obtain a satellite semantic vector map corresponding to the position of the image of the aircraft to be located, and calculate the vector features of the building shape in the satellite semantic vector map to obtain a reference map;

[0046] A matching module is used to perform shape matching on building objects in the building semantic map based on the reference map to obtain a rough matching result, calculate the correlation value of the building objects in the building semantic map corresponding to the rough matching result, and filter the building objects according to the correlation value to obtain a precise matching result;

[0047] The particle filter module is used to perform particle filtering on the precise matching results to obtain the global positioning of the aircraft to be positioned.

[0048] Optionally, in one embodiment of the present application, the particle filter module is specifically configured to:

[0049] Convert all building targets corresponding to the precise matching results in the building semantic graph into particles, where each particle represents a building target, and the attributes of each particle include the absolute position of the particle, the particle weight, and the number of times the particle is successfully matched;

[0050] Initialize all particles and obtain the initialization results;

[0051] Obtain inertial data through the inertial device, perform local mileage propagation on the initialization result based on the particle data and inertial data, and obtain a recursive result, where the recursive result represents the prior state estimate of the particle position at the current moment;

[0052] According to the recursive results, the particle data is measured, updated, and resampled to obtain updated particles;

[0053] The reliability of particles is evaluated according to the number of times each updated particle is successfully matched, and particles that pass the evaluation are defined as valid particles.

[0054] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0056] Figure 1 A schematic flow chart of a large-scale aircraft positioning method based on a satellite vector map provided in Example 1 of the present application;

[0057] Figure 2 This is a flow chart of a large-scale aircraft positioning algorithm based on satellite vector maps according to an embodiment of the present application;

[0058] Figure 3 This is a flow chart of the semantic matching positioning method according to an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a shape information calculation method according to an embodiment of the present application;

[0060] Figure 5 This is a schematic diagram of precise matching in an embodiment of the present application;

[0061] Figure 6 This is a flow chart of the particle filtering process of an embodiment of the present application;

[0062] Figure 7 A schematic structural diagram of a large-scale aircraft positioning device based on satellite vector maps provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0064] The following describes a large-scale aircraft positioning method and apparatus based on a satellite vector map according to an embodiment of the present application with reference to the accompanying drawings.

[0065] Figure 1 A flowchart of a large-scale aircraft positioning method based on satellite vector maps provided in Example 1 of the present application.

[0066] like Figure 1 As shown, the large-scale aircraft positioning method based on satellite vector map includes the following steps:

[0067] Step 101: Acquire an image of the aircraft to be located by using a bird's-eye view camera, and input the image of the aircraft to be located into a semantic neural network to obtain a building semantic map;

[0068] Step 102: Obtain a satellite semantic vector map corresponding to the position of the image of the aircraft to be located, and calculate the vector features of the building shape in the satellite semantic vector map to obtain a reference map;

[0069] Step 103: performing shape matching on building objects in the building semantic map based on the reference map to obtain a rough matching result, calculating correlation values ​​of building objects in the building semantic map corresponding to the rough matching result, and filtering the building objects based on the correlation values ​​to obtain a fine matching result.

[0070] Step 104: perform particle filtering on the precise matching result to obtain the global positioning of the aircraft to be positioned.

[0071] The large-scale aircraft positioning method based on satellite vector maps in the embodiment of the present application uses a downward-looking camera to capture aircraft images, uses satellite maps and captured images for matching and alignment, uses satellite semantic vector maps as matching maps, and combines NCC and shape information to match building semantic maps. After matching, all possible matching results are retained, and particle filtering is used to filter out the correct results, thereby achieving accurate positioning of the aircraft.

[0072] Optionally, in one embodiment of the present application, the vector feature of the building shape in the semantic vector map is a shape feature, and the shape feature is expressed as:

[0073]

[0074] Among them, f A Represents the shape feature vector of point A, s ij Indicates the building area of ​​the region;

[0075] Shape matching uses shape features for matching. The similarity between each two sets of shape features is calculated as follows:

[0076] dis({f A f B f C ...},{f A′ f B′ f C′ ...})=min(dis(f S ,f S′ )),S∈{ABC}

[0077] Among them, f A represents the shape feature vector of point A, and S represents the point set {A, B, C}.

[0078] Optionally, in one embodiment of the present application, the calculation method of the relevant value of the building target is:

[0079]

[0080] Among them, V cam Represents the building outline of the building semantic graph, V ref Represents the building outline of the reference map, V boxRepresents the scope of the building semantic graph.

[0081] Optionally, in one embodiment of the present application, particle filtering is performed on the precise matching result to obtain the global positioning of the aircraft to be positioned, including:

[0082] Convert all building targets corresponding to the precise matching results in the building semantic graph into particles, where each particle represents a building target, and the attributes of each particle include the absolute position of the particle, the particle weight, and the number of times the particle is successfully matched;

[0083] Initialize all particles and obtain the initialization results;

[0084] Obtain inertial data through the inertial device, perform local mileage propagation on the initialization result based on the particle data and inertial data, and obtain a recursive result, where the recursive result represents the prior state estimate of the particle position at the current moment;

[0085] According to the recursive results, the particle data is measured, updated, and resampled to obtain updated particles;

[0086] The reliability of particles is evaluated according to the number of times each updated particle is successfully matched, and particles that pass the evaluation are defined as valid particles.

[0087] Optionally, in one embodiment of the present application, all particles are initialized, which is expressed as:

[0088]

[0089]

[0090] Where m represents the measurement value, f represents the filtered value, 0 represents time 0, θ represents the particle attribute, μ represents the position estimate, Σ represents the variance, α represents the filter weight, + represents the prior, i represents the i-th particle, and c represents the number of successful matches.

[0091] Optionally, in one embodiment of the present application, local mileage propagation is performed on the initialization result based on the aircraft image and inertial data, which is expressed as:

[0092]

[0093] Where t represents time t, f represents the filtered value, μ represents the position estimate, represents the odometer estimate of the relative displacement from time t-1 to time t, Represents variance.

[0094] Optionally, in one embodiment of the present application, the particle data is measured, updated, and resampled according to the recursive result, which is expressed as:

[0095]

[0096]

[0097] Among them, t represents time t, f represents the filter value, Σ represents the variance, + represents the prior, - represents the posterior, α represents the filter weight, and c represents the number of successful matches.

[0098] Optionally, in one embodiment of the present application, the reliability of the particles is evaluated based on the number of times each updated particle is successfully matched, which is expressed as:

[0099]

[0100]

[0101]

[0102] Among them, X t represents the absolute positioning result obtained by particle filtering, S1 and S2 are credibility indicators, m represents the measurement value, t represents time t, f represents the filtered value, c represents the number of successful matches, μ represents the position estimate, α represents the filter weight, and Σ represents the variance.

[0103] Figure 2 This is a diagram of a large-scale aircraft positioning algorithm based on a satellite vector map according to an embodiment of the present application. Figure 2 As shown, inertial data is obtained from the xx device, and the grayscale image is obtained from the downward-looking monocular camera. The local odometry is used to estimate the relative position change between each matching positioning. The camera image is periodically input into the semantic neural network and converted into a semantic building semantic image. Height estimation (ultrasonic measurement or barometer) is required to determine the scale of the downward-looking image. Each match is matched against the entire reference map without odometry constraints. In the pre-processed reference map, the probability distribution of the matching estimated positioning is estimated and used in the initialization and measurement update of the particle filter. After the particles converge, the absolute positioning of the aircraft is estimated.

[0104] Depend on Figure 2 It can be seen that the matching stage includes:

[0105] Building segmentation

[0106] Perform building segmentation on the satellite semantic vector map to obtain a building vector map. Existing neural networks, such as Mask R-CNN, are used for building segmentation.

[0107] Based on the existing OpenStreetMap method, the aircraft image obtained from the overhead camera is converted into a building semantic map.

[0108] Precise and detailed building vector matching

[0109] VMAP has many semantic objects, such as buildings, roads, and water systems. Without loss of generality, we choose to identify buildings as matching targets here. If you want to match other semantic objects, such as roads, water systems, playgrounds, windmills, etc., you only need to add the corresponding matching function.

[0110] Semantic building coarse-to-fine search methods such as Figure 3 The initial position needs to be within 10000km 2 The range is estimated, and the corresponding range VMAP is converted into a semantic segmentation map as a reference map. Shape features are calculated for each point on the map.

[0111] First, for each building, calculate the shape features. Search for targets with a similarity of more than 50% based on the shape features as the rough matching results.

[0112] Then perform fine matching.

[0113] Figure 4 The shape information calculation method diagram is as follows: Figure 4 As shown in the figure, white is a building. After encoding each image block, its shape information relative to the center of the image is calculated.

[0114] The shape feature is expressed as:

[0115]

[0116] Among them, f A Represents the shape feature vector of point A, s ij express Figure 4 The building area of ​​the area shown,

[0117] The shape feature similarity description method is:

[0118]

[0119] Considering that multiple points can be selected for each building to calculate the shape features, the similarity of two sets of shape features is defined as:

[0120] dis({f A f B f C ...},{f A′ f B′ f C′ ...})=min(dis(f S ,f S′ )),S∈{ABC}

[0121] Among them, A, B, C are a set of points, each of which has a corresponding shape vector f A 、fB 、f C , A', B', C' are another set of points, each of which also has a corresponding shape vector f A '、f B 、f C , the above formula gives the calculation method for finding the similarity of these two groups of points, S represents the point set {A, B, C...}.

[0122] The fine matching scans around the rough matching results and takes the result with the highest correlation value. Figure 5 is a schematic diagram of precise matching, such as Figure 5 As shown, the yellow box represents the aerial image range V box , the blue outline is the reference building outline V ref , the green outline is the building outline in the aerial photo V cam .

[0123] Define the exact matching correlation value as:

[0124]

[0125] Among them, intersection is an operation to calculate the overlapping area of ​​graphics, which can be implemented by directly calling the Python graphics library. The correlation value is the similarity used in precise matching. The correlation value used in precise matching is similar to the similarity, but with higher accuracy and slower calculation speed.

[0126] Particle filter stage

[0127]

[0128] Table 1 Variable list

[0129] Table 1 shows the symbols used in the calculation of the global rate distribution. There are N particles in total. For each particle p, the attributes include is a 2D vector representing the absolute position of the particle, is the variance, is the particle weight, is the number of times a particle is successfully matched. Each particle is considered a match if its distance to the nearest matching result is less than δ. The subscripts t and i represent particle i at time t.

[0130] Each match has multiple matching results. For each matching position, the attributes include in is a 2D vector representing the absolute position of the matching solution, is the estimated similarity. The mixture distribution is:

[0131]

[0132]

[0133] θ k =(α k ,μ k ,Σ k )

[0134] The incorrect matching positioning results are randomly distributed, while the correct matching positioning results are constrained by the motion trajectory. Sequential navigation between adjacent matching positioning can estimate the relative position relationship between adjacent matching results. As the number of matches increases, the estimation of the motion trajectory based on sequential navigation can eliminate incorrect matching results and find the correct matching results. Based on this idea, a particle filter is established to find the correct matching results. The particle filter algorithm is as follows: Figure 6 The particle filter is a sequential Bayesian filter that consists of four steps, where steps 2-4 are executed repeatedly.

[0135] Depend on Figure 6 It can be seen that particle filtering includes:

[0136] initialization

[0137] The system equations and measurement equations are:

[0138]

[0139]

[0140] Among them, the superscript m represents the measurement value, f represents the filtered value, v represents the odometer estimate, the subscripts 0 and t, t-1 represent the number of filtering steps, the superscript - represents the prior, + represents the posterior, the subscript i represents the i-th particle, θ is the particle attribute, μ represents the position estimate, represents the variance, Represents the odometry estimate of the relative displacement from time t-1 to time t, for example, Represents the prior position estimate of the i-th particle at time t.

[0141] Local Mileage Propagation

[0142] Recursion

[0143] At each moment, the particle position needs to be estimated a priori. The particle is recursively calculated using the following dynamic equation.

[0144]

[0145] Where t represents time t, f represents the filtered value, μ represents the position estimate, represents the odometer estimate of the relative displacement from time t-1 to time t, represents the variance, for example, Represents the posterior method of the i-th particle at time t-1.

[0146] Measurement Updates and Resampling

[0147] The semantic matching positioning result is used to update the measurement. The semantic matching result is a distribution of the entire map.

[0148] Each particle i is assigned a weight at time t and the number of successful matches See the following formula for details:

[0149]

[0150]

[0151] Normalization is needed for resampling, in is rounded down.

[0152] Reliability assessment

[0153] The particles with more than n successful matches (n=3 here) are defined as valid particles. Particle filtering obtains the absolute positioning result X t , and the credibility indicators S1 and S2 are shown in the following formula:

[0154]

[0155]

[0156]

[0157] Among them, X t represents the absolute positioning result obtained by particle filtering, S1 and S2 are credibility indicators, m represents the measurement value, t represents time t, f represents the filter value, c represents the number of successful matches, μ represents the position estimate, α represents the filter weight, and Σ represents the variance. For example, Represents the filter weight of the i-th particle at time t.

[0158] In order to implement the above embodiment, the present application also proposes a large-scale aircraft positioning device based on satellite vector maps.

[0159] Figure 7 A schematic structural diagram of a large-scale aircraft positioning device based on satellite vector maps provided in an embodiment of the present application.

[0160] like Figure 7 As shown, the large-scale aircraft positioning device based on satellite vector map includes a to-be-positioned map acquisition module, a reference map acquisition module, a matching module, and a particle filter module, wherein:

[0161] The module for acquiring the image of the aircraft to be located uses a bird's-eye view camera to obtain the image of the aircraft to be located, and inputs the image of the aircraft to be located into the semantic neural network to obtain a semantic image of the building;

[0162] A reference map acquisition module is used to obtain a satellite semantic vector map corresponding to the position of the image of the aircraft to be located, and calculate the vector features of the building shape in the satellite semantic vector map to obtain a reference map;

[0163] A matching module is used to perform shape matching on building objects in the building semantic map based on the reference map to obtain a rough matching result, calculate the correlation value of the building objects in the building semantic map corresponding to the rough matching result, and filter the building objects according to the correlation value to obtain a precise matching result;

[0164] The particle filter module is used to perform particle filtering on the precise matching results to obtain the global positioning of the aircraft to be positioned.

[0165] Optionally, in one embodiment of the present application, the particle filter module is specifically configured to:

[0166] Convert all building targets corresponding to the precise matching results in the building semantic graph into particles, where each particle represents a building target, and the attributes of each particle include the absolute position of the particle, the particle weight, and the number of times the particle is successfully matched;

[0167] Initialize all particles and obtain the initialization results;

[0168] Obtain inertial data through the inertial device, perform local mileage propagation on the initialization result based on the particle data and inertial data, and obtain a recursive result, where the recursive result represents the prior state estimate of the particle position at the current moment;

[0169] According to the recursive results, the particle data is measured, updated, and resampled to obtain updated particles;

[0170] The reliability of particles is evaluated according to the number of times each updated particle is successfully matched, and particles that pass the evaluation are defined as valid particles.

[0171] It should be noted that the aforementioned explanation of the embodiment of the large-scale aircraft positioning method based on satellite vector maps is also applicable to the large-scale aircraft positioning device based on satellite vector maps in this embodiment, and will not be repeated here.

[0172] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0173] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0174] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0175] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0176] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0177] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0178] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0179] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A large-scale aircraft positioning method based on satellite vector maps, characterized in that: The following steps are involved: Acquire an image of the aircraft to be located by a bird's-eye view camera, and input the image of the aircraft to be located into a semantic neural network to obtain a semantic map of the building; Obtaining a satellite semantic vector map of a position corresponding to the image of the aircraft to be located, and calculating vector features of building shapes in the satellite semantic vector map to obtain a reference map; Performing shape matching on building objects in the building semantic map based on the reference map to obtain a rough matching result, calculating correlation values ​​of building objects in the building semantic map corresponding to the rough matching result, and filtering building objects according to the correlation values ​​to obtain a precise matching result; Performing particle filtering on the precise matching result to obtain the global positioning of the aircraft to be positioned; The vector features of the building shapes in the semantic vector map are shape features, which are expressed as: Among them, f A Represents the shape feature vector of point A, s ij Indicates the building area of ​​the region; The shape matching is performed using shape features, and the similarity between each two sets of shape features is calculated as follows: dis({f A f B f C ...},{f A′ f B′ f C′ ...})=min(dis(f S ,f S′ )),S∈{ABC} Among them, f A represents the shape feature vector of point A, S represents the point set {A, B, C}, and A, B, C represent a set of points on the building in the semantic vector map; The relevant value of the building target is calculated as follows: Among them, V cam Represents the building outline of the building semantic graph, V ref Represents the building outline of the reference map, V box It represents the scope of the building semantic graph, and intersection is an operation to find the overlapping area of ​​the graphics.

2. The method according to claim 1, wherein The performing of particle filtering on the precise matching result to obtain the global positioning of the aircraft to be positioned includes: Convert all building targets corresponding to the precise matching results in the building semantic graph into particles, wherein each particle represents a building target, and the attributes of each particle include the absolute position of the particle, the particle weight, and the number of times the particle is successfully matched; Initialize all particles and obtain the initialization results; Acquire inertial data through an inertial device, perform local mileage propagation on the initialization result based on the particle data and the inertial data, and obtain a recursive result, wherein the recursive result represents a priori state estimate of the particle position at the current moment; According to the recursive result, the data of the particle is measured, updated, and resampled to obtain an updated particle; The reliability of particles is evaluated according to the number of times each updated particle is successfully matched, and particles that pass the evaluation are defined as valid particles.

3. The method according to claim 2, wherein The initialization of all particles is expressed as: Where m represents the measurement value, f represents the filtered value, 0 represents time 0, θ represents the particle attribute, μ represents the position estimate, Σ represents the variance, α represents the filter weight, + represents the prior, i represents the i-th particle, and c represents the number of successful matches.

4. The method according to claim 3, wherein The local mileage propagation of the initialization result based on the aircraft image and the inertial data is expressed as: Where t represents time t, f represents the filtered value, μ represents the position estimate, represents the odometer estimate of the relative displacement from time t-1 to time t, Represents variance.

5. The method according to claim 4, wherein The data of the particle is measured, updated, and resampled according to the recursive result, which is expressed as: Among them, t represents time t, f represents the filter value, Σ represents the variance, + represents the prior, - represents the posterior, α represents the filter weight, and c represents the number of successful matches.

6. The method according to claim 5, wherein The reliability of the particles is evaluated based on the number of times each updated particle is successfully matched, which is expressed as: Among them, X t represents the absolute positioning result obtained by particle filtering, S1 and S2 are credibility indicators, m represents the measurement value, t represents time t, f represents the filtered value, c represents the number of successful matches, μ represents the position estimate, α represents the filter weight, and Σ represents the variance.

7. A large-scale aircraft positioning device based on satellite vector maps, characterized in that: It includes a map acquisition module to be located, a reference map acquisition module, a matching module, and a particle filter module, wherein: The image acquisition module for the object to be located acquires an image of the object to be located through a bird's-eye view camera, and inputs the image of the object to be located into a semantic neural network to obtain a semantic image of the building; The reference map acquisition module is used to obtain a satellite semantic vector map of the position corresponding to the image of the aircraft to be located, and calculate the vector features of the building shape in the satellite semantic vector map to obtain a reference map; The matching module is configured to perform shape matching on building objects in the building semantic map based on the reference map to obtain a rough matching result, calculate correlation values ​​of building objects in the building semantic map corresponding to the rough matching result, and filter building objects based on the correlation values ​​to obtain a precise matching result; The particle filtering module is used to perform particle filtering on the precise matching result to obtain the global positioning of the aircraft to be positioned; The vector features of the building shapes in the semantic vector map are shape features, which are expressed as: Among them, f A Represents the shape feature vector of point A, s ij Indicates the building area of ​​the region; The shape matching is performed using shape features, and the similarity between each two sets of shape features is calculated as follows: dis({f A f B f C ...},{f A′ f B′ f C′ ...})=min(dis(f S ,f S′ )),S∈{ABC} Among them, f A represents the shape feature vector of point A, S represents the point set {A, B, C}, and A, B, C represent a set of points on the building in the semantic vector map; The relevant value of the building target is calculated as follows: Among them, V cam Represents the building outline of the building semantic graph, V ref Represents the building outline of the reference map, V box It represents the scope of the building semantic graph, and intersection is an operation to find the overlapping area of ​​the graphics.

8. The device according to claim 7, wherein The particle filter module is specifically used to: Convert all building targets corresponding to the precise matching results in the building semantic graph into particles, wherein each particle represents a building target, and the attributes of each particle include the absolute position of the particle, the particle weight, and the number of times the particle is successfully matched; Initialize all particles and obtain the initialization results; Acquire inertial data through an inertial device, perform local mileage propagation on the initialization result based on particle data and the inertial data, and obtain a recursive result, wherein the recursive result represents a priori state estimate of the particle position at the current moment; According to the recursive result, the data of the particle is measured, updated, and resampled to obtain an updated particle; The reliability of particles is evaluated according to the number of times each updated particle is successfully matched, and particles that pass the evaluation are defined as valid particles.

Images