Positioning method, device and system based on visual image and storage medium

Abstract

The invention provides a positioning method, device and system based on a visual image and a storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring earth surface feature information based on a to-be-positioned image acquired by observing a to-be-positioned area by a user terminal; matching a contour line corresponding to the earth surface feature information and geographic position information of each contour point on the contour line from an earth surface form library; and determining the pose information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the to-be-positioned image and the geographic position information of the target contour point. According to the invention, the earth surface image and form information when the to-be-positioned area is the satellite navigation denial area can be rapidly obtained by using space remote sensing, and the information is matched with the earth surface image, so that positioning efficiency and precision are taken into account while positioning and orienting are carried out in the satellite navigation denial area for individuals, unmanned aerial vehicles or light carriers.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a positioning method, device, system and storage medium based on visual images. Background Art

[0002] Positioning technology is one of the indispensable technologies for modern national defense and warfare. At present, the commonly used positioning methods mainly include satellite positioning, communication base station positioning, WIFI node positioning and Bluetooth positioning. However, all of the above methods rely on third-party servers or base stations for auxiliary positioning, and cannot be positioned only through the terminal's own environmental information. If working in an area with signal shielding in the wild or a modern war occurs, the satellite will be the first to be shot down, and the base station will also be destroyed by artillery fire. At this time, the above methods will not be able to effectively locate.

[0003] Therefore, in a satellite-denied (GNSSDenied) environment, positioning technology based on visual images is usually used for positioning. Its classic process includes: first, a large number of overlapping photos are collected for the positioning field, and the significant image feature points of the positioning field are extracted. The object coordinates of the significant image feature points in the positioning field are determined by using the principles of density matching and motion recovery structure, and a feature point cloud library is established. During positioning, the feature points of the positioning image are calculated and matched with the image features in the image feature point cloud library, and the known object coordinates of the matching feature points are used for intersection to determine the position of the camera and the posture of the mobile phone when shooting. It can be seen that the feature point cloud library is the basis of visual image positioning. However, it is difficult to obtain ground or low-altitude perspective images abroad or in other non-cooperative areas, which greatly limits the scope of application of traditional visual image positioning. Therefore, the efficiency and accuracy of visual image positioning performed in a satellite-denied environment are relatively low. Summary of the invention

[0004] The present invention provides a positioning method, device, system and storage medium based on visual images, which are used to solve the defects of low efficiency and accuracy of visual image positioning performed in a satellite denial environment in the prior art.

[0005] The present invention provides a positioning method based on visual images, comprising:

[0006] Acquire surface feature information based on the image to be located collected by the user terminal through observation of the area to be located;

[0007] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the positioning method based on visual images as described above is implemented.

[0008] The positioning method, device, system and storage medium based on visual images provided by the present invention, after the user terminal shoots the image to be positioned in the area to be positioned, uses the surface feature information extracted from the image to query the matching contour line and the geographical location area where it is located in the surface morphology library, and then uses the geographical location information and pixel location information of multiple target contour points in the geographical location area to perform fine positioning, and obtain the position and posture of the user terminal when shooting the image to be positioned. It is possible to use aerospace remote sensing to quickly obtain the surface image and morphological information when the area to be positioned is a satellite navigation denied area, and match this information with the ground image to achieve positioning and orientation for individuals, drones or light vehicles in the satellite navigation denied area while taking into account positioning efficiency and accuracy, which can avoid the limitations of GNSS interference, magnetic field positioning requiring the acquisition of relevant data in advance, radio positioning requiring the construction of a large number of new infrastructure, and traditional visual positioning requiring the construction of feature point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 It is a flow chart of the positioning method based on visual images provided by the present invention;

[0011] Figure 2 It is a schematic diagram of visual area calculation provided by the present invention;

[0012] Figure 3 It is a schematic diagram of the visual field shape provided by the present invention;

[0013] Figure 4 is a structural schematic diagram of a positioning device based on visual images provided by the present invention;

[0014] Figure 5 It is a structural schematic diagram of a positioning system based on visual images provided by the present invention;

[0015] Figure 6 It is a schematic diagram of the flow of the positioning system based on visual images provided by the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0018] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present invention, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0019] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0020] Figure 1 FIG. 1 is a flow chart of the positioning method based on visual images provided by the present invention. Figure 1 As shown, the visual image-based positioning method provided by an embodiment of the present invention includes: step 101, acquiring surface feature information based on the image to be positioned collected by the user terminal through observation of the area to be positioned.

[0021] It should be noted that the executor of the positioning method based on visual images is a positioning device based on visual images, which can be integrated on a control chip in any user terminal in the area to be positioned.

[0022] The area to be positioned described in the embodiment of the present invention may include a satellite navigation denied area and a non-satellite navigation denied area.

[0023] One application scenario of the visual image-based positioning method provided by an embodiment of the present invention is to use a camera arranged in a user terminal to capture scenes in a satellite navigation denied area to form an image to be positioned, and to match the surface contour features contained in the image against a surface morphology library to obtain the actual geographical location of the surface contour features, and then use the actual geographical location on the contour to locate and orient the user terminal that captured the image to be positioned.

[0024] Among them, the satellite navigation denial area refers to an environment where the satellite navigation system cannot be used due to natural or human factors, but the WIFI, Bluetooth and other wireless networks between devices in this environment are not affected by signal attenuation, environmental interference, and multipath propagation.

[0025] Specifically, in step 101, the positioning device based on visual images uses the image obtained by the user terminal taking pictures of the scene in the satellite navigation denied area as the image to be positioned, and uses the image processing algorithm to extract the surface feature information for characterizing the topography from the image to be positioned.

[0026] Step 102: Using the surface feature information, a contour line corresponding to the surface feature information and the geographical location information of each contour point on the contour line are matched from a surface morphology library.

[0027] The surface morphology library is a spatial digital model of buildings and / or skylines constructed from terrain data obtained by ground observation by an aerial sensing device in advance. The aerial sensing device includes one or more of a high-resolution satellite, a camera or a radar carried by an unmanned aerial vehicle.

[0028] It should be noted that before step 102, since the communication of the satellite navigation system is cut off in the satellite navigation denied area, the area can only be observed by relying on high-resolution satellites and aerial perception devices such as cameras or radars carried by drones, and the aerospace images collected during its cruise are transmitted to the positioning device based on visual images, and the terrain data extracted therefrom are used to construct a surface morphology library corresponding to the area.

[0029] The surface morphology library is a spatial digital model corresponding to multiple satellite navigation denied areas. Each spatial digital model retains at least the surveying and mapping data corresponding to contour features such as buildings and skylines.

[0030] Among them, the types of spatial digital models include but are not limited to digital elevation model (Digital Elevation Model, DEM), digital surface model (Digital Surface Model, DSM), digital orthophoto map (Digital Orthophoto Map, DOM), true orthophoto (True Orho / True Digital Ortho Map, TDOM), oblique photography 3D model, laser point cloud, etc., and the embodiments of the present invention do not make specific limitations on this.

[0031] It is understandable that when the aviation perception device is a drone carrier, it can also be equipped with a terminal connected to the camera or radar communication, so as to establish a spatial digital model based on the aerospace images received in real time during the drone cruising process, and then transmit the spatial digital model directly to the positioning device based on visual images.

[0032] Specifically, in step 102, the positioning device based on visual images uses the surface feature information obtained in step 101 as retrieval information to perform query matching in the surface morphology library, and the results are divided into two types: successful matching and failed matching.

[0033] A successful match means that the surface feature information contained in the current user's shooting picture can match the information stored in the locally stored surface morphology library, which means that the terrain features of the area have been included. Therefore, the contour line corresponding to the surface feature information obtained by shooting at a certain location in the current area can be extracted from the surface morphology library, and the geographical location information of each contour point on the contour line can be extracted from the spatial digital model.

[0034] Matching failure means that the surface feature information contained in the current user's shooting picture cannot match any information in the locally stored surface morphology library, which means that the aerial sensing device has not conducted any observation activities in the area. Therefore, it is necessary to first use the aerial sensing device to observe the current area and then construct the corresponding spatial digital model, and then carry out query and matching work on the surface feature information based on the obtained spatial digital model.

[0035] Step 103: Determine the position information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the image to be located and the geographical location information of the target contour point.

[0036] The target contour point is any contour point on the contour line.

[0037] Specifically, in step 103, the positioning device based on visual images extracts at least three or more contour points from multiple contour points on the contour line as target contour points, applies them to the rear intersection principle, and calculates the posture information of the user terminal as the photographer in the geodetic coordinate system.

[0038] After the user terminal shoots the image to be located in the area to be located, the embodiment of the present invention uses the surface feature information extracted from the image to query the matching contour line and the geographical location area where it is located in the surface morphology library, and then uses the geographical location information and pixel location information of multiple target contour points in the geographical location area for fine positioning to obtain the position and posture of the user terminal when shooting the image to be located. It is possible to use aerospace remote sensing to quickly obtain surface images and morphological information when the area to be located is a satellite navigation denied area, and match this information with the ground image to achieve positioning and orientation for individuals, drones or light vehicles in the satellite navigation denied area while taking into account positioning efficiency and accuracy, which can avoid GNSS interference, magnetic field positioning requires the acquisition of relevant data in advance, radio positioning requires the construction of a large number of new infrastructure, and traditional visual positioning requires the construction of feature point clouds.

[0039] Based on any of the above embodiments, before obtaining the surface feature information from the image to be located collected by the user terminal based on the observation of the area to be located, it also includes: establishing a grid according to the degree of change of the terrain data obtained by the aerial sensing device from the ground observation.

[0040] Specifically, before step 102, the positioning device based on visual images uses photogrammetry to establish a route model or a regional network model corresponding to the actual situation according to the terrain data obtained by the aerial perception device from the ground observation.

[0041] Among them, the regional grid model can divide the regional space into regular or irregular grid cells, and each grid cell corresponds to an elevation data. Mathematically, it can be expressed as a matrix, and in computer implementation, it is a two-dimensional array. Each grid cell or an element of the array corresponds to an elevation value.

[0042] Optionally, if the aerial perception device is a high-resolution satellite, a mesh model is constructed by dense matching based on two sets of stereo satellite image data.

[0043] In the image correction stage, correction is performed based on the satellite orbit parameter RPC. For the correction of a single image, two local control points are introduced to participate in the correction to obtain the correct position coordinate information of each pixel.

[0044] During the aerial triangulation stage, by loading the reference terrain and integrating the visual and spatial information of the photos, the images in the survey area are projected onto the reference terrain, and the survey area is automatically split according to the geographic coordinates. The blocks are quickly traversed and calculated by alternating between Incremental and Global matching strategies to correct the image position and posture. A joint adjustment is performed based on the calculated posture of the blocks to obtain high-precision results.

[0045] In the modeling stage, the generation of digital elevation model (DEM) and digital surface model (DSM) is based on K-means clustering and RBFNN construction method. DEM can be constructed using the generated point cloud data, and the ground point cloud filtered by this method has a good effect on elevation prediction. Using the SGM pixel-level dense matching algorithm, DSM can be automatically generated, and some erroneous DEM and DSM are corrected by manual editing.

[0046] Optionally, if the aerial perception device is a drone platform, the process of generating a digital elevation model (DEM) based on the airborne lidar point cloud data generally has three important steps: point cloud denoising, point cloud filtering, and point cloud interpolation to generate DEM.

[0047] Firstly, the point cloud noise is removed based on the statistical filtering and radius filtering algorithms.

[0048] Secondly, the objects are classified according to the elevation information of the surrounding point cloud data. The airborne laser radar point cloud data is composed of three-dimensional discrete points, which does not contain the attribute information of the objects. However, we can judge based on the spatial distribution relationship of other point clouds in a certain area, and can organize and summarize the spatial correlation of the types of objects in the point cloud data, so that the ground point cloud data can be distinguished. The project plans to adopt an improved cloth simulation point cloud filtering algorithm based on scene differentiation. The algorithm first identifies the scene information and then uses adaptive adjustment of the filtering parameters. The algorithm can not only reduce the degree of manual intervention in the filtering process, but also accurately extract the ground points.

[0049] Finally, DEM is generated.

[0050] There will be some blank areas in the filtered ground point cloud data, which are called data holes. This is because the data left on the ground by filtering out buildings, trees and other objects that are not ground points is missing during the point cloud data filtering process. To generate a digital elevation model from ground point cloud data, it is necessary to reconstruct the data of the filtered ground discrete points. This process is point cloud data interpolation, which requires interpolation operations based on the elevation of the adjacent point cloud data or the characteristic values ​​of the terrain ups and downs.

[0051] The skyline contour features calculated from the elevation data in each grid are used as the contour line index of the corresponding grid to generate the surface morphology library.

[0052] Specifically, the positioning device based on visual images performs skyline detection on the elevation data in each grid, and stores the detected skyline contour features as the contour line index of the corresponding grid in the surface morphology library, so that when a query is performed in the surface morphology library, the grid in which it is located can be roughly located by matching the surface feature information with the contour line index.

[0053] There are two different strategies for skyline detection: one is the skyline detection method based on OpenGL rendering, which achieves the same effect as real shooting by setting parameters such as the height, focal length and field of view of the virtual camera, and extracts the corresponding skyline in the window, but it requires rendering the data, which is computationally intensive, time-consuming and computationally resource-intensive; the other is to detect the skyline by calculating visible points and transforming coordinate projections, using the ray method to find visible points, and then coordinate projecting these visible points to obtain the skyline. Since the effective sampling points per unit angle in the area close to the observation point are sparse, and the effective sampling points per unit angle in the area far from the observation point are dense, using this method to detect the skyline will result in inconsistent resolution, which will have a significant impact on the normalization, feature extraction and matching of the skyline.

[0054] Preferably, Figure 2 This is a schematic diagram of visual area calculation provided by the present invention. Figure 3 Schematic diagram of the visual field shape provided by the present invention. Figure 2 and 3 As shown, the skyline detection using elevation data in the embodiment of the present invention is mainly divided into two steps: visible area calculation and coordinate projection transformation.

[0055] Step 1: Calculate the visible area. With the observation point as the center, divide the elevation data into 8 quadrants, and calculate the visible points in each quadrant according to the visual angle relationship. Here we take the first quadrant as an example. Figure 2 shown.

[0056] First, raise the height of point O by 1.5m (simulating the height of a person or the height of a camera). The line connecting points P1 and P2 may block point A. Based on the height relationship of these three points, determine whether point A is a visible point. If it is, keep the original height of point A. If not, update the height of point A to the minimum visible height. The same is true for points B and C.

[0057] The update of height is mainly for the convenience of judging the sampling points later. At the same time, the sampling points on the minimum circumscribed circle of the visible domain can be used to replace the outermost contour of the visible domain for coordinate projection. The other sampling points in the first quadrant use the same method to judge whether they are visible and decide whether to update the height.

[0058] Step 2: Coordinate projection transformation. All sampling points constitute the visual domain of the observation point, but the shape of the visual domain is usually irregular. If the coordinate projection transformation is performed using the outermost points of the visual domain, inconsistent resolution will occur, such as Figure 3 shown.

[0059] Figure 3 The number of valid sampling points between the line connecting points a and b is less than the number of valid sampling points between the line connecting points b and c. Figure 3 The projection transformation is performed on the points A′, B′, and C′ in the figure. Because the heights of the invisible points are updated when calculating the visible area, it is equivalent to using the points on the minimum circumscribed circle and the points on the visible area contour for projection.

[0060] The embodiment of the present invention establishes a grid in advance according to the severity of changes in terrain data obtained by ground observation by an aerial sensing device, and extracts skyline outline features from each grid as corresponding coarse-grained contour line indexes to form a surface morphology library. It is possible to quickly achieve low-precision positioning by matching the skyline outline extracted from the image with the coarse matching feature index of the grid, and then perform high-precision positioning within the grid and with the refined ratio of the surface morphology library, thereby improving positioning efficiency.

[0061] Based on any of the above embodiments, the surface feature information includes skyline feature information.

[0062] Accordingly, the acquiring of surface feature information based on the image to be located collected by the user terminal through observation of the area to be located includes: inputting the image to be located into a skyline detection model, and acquiring a skyline probability map output by the skyline detection model.

[0063] The skyline detection model is trained based on sample remote sensing images and skyline labels annotated corresponding to the sample remote sensing images.

[0064] It should be noted that the skyline detection model is obtained after training based on remote sensing image sample data and skyline labels pre-annotated in remote sensing images.

[0065] The skyline detection model may be an artificial intelligence model, and the embodiment of the present invention does not specifically limit the model type.

[0066] For example, the skyline detection model is mainly composed of two parts: the encoder and the decoder. The encoder is responsible for extracting features, and the decoder is responsible for predicting probability maps. Semantic segmentation can extract richer global information through multi-layer convolution to adapt to the feature distribution of different samples, and has strong adaptability and generalization capabilities.

[0067] It should be noted that the sample data includes remote sensing image data and skyline labels corresponding to the sample data. The remote sensing image sample data is divided into a training set and a test set, and the embodiment of the present invention does not specifically limit the sample ratio of the training set and the test set.

[0068] Specifically, in step 101, the surface feature information extracted by the positioning device based on visual images from the image to be positioned may be skyline feature information, and the following steps are performed when extracting the feature:

[0069] First, the positioning device based on visual images sets the skyline detection model according to the trained model parameters, and then uses the model to extract the skyline contour of any image to be positioned in step 101 to obtain a skyline probability map corresponding to the image to be positioned.

[0070] Among them, the skyline probability map is a two-dimensional array with the same width and height as the image to be located. The value range of any value in the array is [0, 1], where 0 means there is no cloud at the pixel point and 1 means there is cloud at the pixel point. In addition, the position information of the pixel points in the two-dimensional array can be used to know which areas in the image are clouds.

[0071] By performing dynamic programming on the skyline probability graph, the longest path formed by connecting points one by one is used as the skyline feature information.

[0072] Specifically, the positioning device based on visual images is based on the skyline probability map output by the skyline detection model, uses dynamic programming to connect the probability map, and outputs the longest path formed by the connection as the skyline feature information.

[0073] Preferably, in dynamic programming, the probability of the points on the connecting line is used as the objective function, and the span of the pixels between two adjacent columns is used as a constraint. From step N to step N+1, it is necessary not only to find the point with the highest probability in the next column, but also to ensure that the difference in the vertical coordinates of the pixels in the two adjacent columns is within the allowable range, and to optimize the objective function as much as possible.

[0074] The embodiment of the present invention classifies the graphic area through the skyline detection model to obtain a skyline probability map, and uses dynamic programming to connect the intermittent boundary lines in the skyline probability map, so as to find a longest boundary that can be connected from left to right as the skyline. The color information and gradient changes between two adjacent columns of pixels can be comprehensively considered at each step of the dynamic programming search to reduce the situation of skyline breaks, thereby ensuring a better image skyline detection effect and improving the positioning accuracy based on visual images.

[0075] Based on any of the above embodiments, the method of using surface feature information to match a contour line corresponding to the surface feature information and the geographic location information of each contour point on the contour line from a surface morphology library includes: if it is determined that the area to be positioned is an urban environment, after querying a first target grid from the surface morphology library using the first skyline feature information as an index, obtaining the geographic location information of each contour point on the contour line based on the first target grid.

[0076] If it is determined that the satellite navigation denied area to be positioned is not an urban environment, the second skyline feature information and elevation data are used as an index to query the second target grid from the surface morphology library, and then an equal-interval sampling serialization method is used in the second target grid to obtain the geographic location information of each contour point on the contour line.

[0077] The first skyline feature information is a panoramic skyline extracted from an image of a city scene taken vertically upward by a user terminal, and the second skyline feature information is a skyline extracted from an image of a non-city scene taken by a user terminal.

[0078] Specifically, the positioning device based on visual images can divide the use environment of the satellite navigation denied area to be positioned into an urban environment and a non-urban environment (including the use environment of Mars, etc.), so as to implement different grid positioning schemes in different use environments:

[0079] For urban environments, the positioning device based on visual images can use the first skyline feature information to match the contour line index represented by the skyline contour feature in the surface morphology library to obtain the first target grid where it is located, and use the corresponding skyline contour feature stored in the first target grid as the contour line to obtain the geographic location information of each contour point therein.

[0080] The first skyline feature information may be skyline features obtained from an image to be positioned shot in a city scene, or a panoramic skyline extracted from a city scene image shot vertically upward in a city by a user terminal.

[0081] It can be understood that the skyline detection method for this usage scenario can be divided into two different situations. One is to use the graph cutting method to detect the skyline in clear weather, and the other is to use the shortest path method to detect the skyline at night, in fog or in obstructions.

[0082] For non-urban environments, the positioning device based on visual images uses a combination of the second skyline feature information and elevation data to match the contour line index in the surface morphology library to obtain the second target grid where it is located, and uses an equally spaced sampling serialization method within the second target grid to represent the contour line corresponding to the skyline to obtain the geographic location information of each contour point therein.

[0083] For example, eight points are sampled at equal intervals in each sliding window of the second target grid, and then serialized according to the relative heights of the sampling points. Each point is represented by a three-bit binary number, and eight points form a 24-bit contour word representing the characteristics of the skyline in the sliding window.

[0084] The second skyline feature information is a skyline feature obtained from an image to be positioned taken in a non-urban scene.

[0085] According to different scenarios corresponding to the satellite navigation denied area to be positioned, the embodiment of the present invention decides to use different indexes to match in the surface morphology library, so as to use the corresponding skyline representation method in the matched target grid to obtain the geographical location information of each contour point on the skyline. It can take into account a variety of usage scenarios on the basis of improving the positioning accuracy based on visual images, and improve the scalability of positioning based on visual images.

[0086] On the basis of any of the above embodiments, before obtaining the surface feature information from the image to be positioned collected by the user terminal based on the observation of the navigation denied area of ​​the satellite to be positioned, it also includes: establishing a grid according to the degree of change of the terrain data obtained by the aerial sensing device from the ground observation.

[0087] Specifically, before step 102, the positioning device based on visual images uses photogrammetry to establish a route model or a regional network model corresponding to the actual situation according to the terrain data obtained by the aerial perception device from the ground observation.

[0088] Among them, the regional grid model can divide the regional space into regular or irregular grid cells, and each grid cell corresponds to an elevation data. Mathematically, it can be expressed as a matrix, and in computer implementation, it is a two-dimensional array. Each grid cell or an element of the array corresponds to an elevation value.

[0089] Furthermore, in each grid, the building outline features extracted from the building facades are used as the contour line index of the corresponding grid by taking the superpixel block as the processing unit to generate the surface morphology library.

[0090] Specifically, the positioning device based on visual images performs building detection on the elevation data within each grid, and stores the detected building outline features as the contour line index of the corresponding grid in the surface morphology library, so that when a query is performed in the surface morphology library, the grid where the building is located can be roughly located by matching the surface feature information with the contour line index.

[0091] Among them, there are three main types of building outline information extraction methods:

[0092] (1) Pre-process a batch of common templates containing targets, use prior target information, and adopt template matching to extract target contours. This method is simple to implement, but has many limitations for extracting buildings and has limited extraction accuracy.

[0093] (2) Combining edge detection and line extraction methods, in building images, due to the complex background and distortion caused by the imaging process, various edge detection algorithms cannot accurately distinguish all building edges, making it difficult to automatically extract all contours.

[0094] (3) A method of extracting the target contour by combining the active contour model. The active contour model (Snake model) assigns an energy function to the curve based on the existing initial contour curve. By minimizing the energy function, the curve evolves toward the true contour. When the energy is minimized, the position of the true contour curve is obtained. The Snake model is difficult to implement and has a very poor effect on concave segmentation, which often occurs in buildings, so the Snake model cannot be used directly.

[0095] Preferably, the embodiment of the present invention uses a small area composed of pixels with adjacent positions and similar characteristics such as color, brightness, texture, etc. in the image to be located as a superpixel block. A fast building facade image contour extraction algorithm is combined with a level set evolution algorithm using superpixel blocks as processing units.

[0096] The embodiment of the present invention establishes a grid in advance according to the severity of the change in terrain data obtained by the aerial sensing device from the ground observation, and extracts the building outline features from the building facades in each grid as the corresponding coarse-grained contour line index to form a surface morphology library. It is possible to quickly achieve low-precision positioning by matching the building outline extracted from the image with the coarse matching feature index of the grid, and then perform high-precision positioning within the grid and the surface morphology library in a refined manner, thereby improving positioning efficiency.

[0097] Based on any of the above embodiments, the surface feature information includes building outline feature information;

[0098] Accordingly, based on the image to be located collected by the user terminal through observation of the area to be located, the surface feature information is obtained, including:

[0099] Extract the saliency map of the image to be located, and use the superpixel block algorithm to divide the saliency map into blocks to generate a saliency block map;

[0100] Based on the texture features of the building, the building blocks are extracted from the significant block map, and the building blocks are merged to obtain the initial outline of the building;

[0101] The level set method is used to perform evolution calculation on the initial building outline to obtain the building outline feature information.

[0102] It should be noted that a saliency map is a pattern of image partitioning, which is an image that shows the uniqueness of each pixel in the image to be located. Its goal is to simplify or change the representation of a general image into a style that is easier to analyze.

[0103] In this embodiment, the salient block map refers to an image obtained by performing super-pixel block division on the salient map of the image to be located.

[0104] Specifically, in the embodiment of the present invention, the Quaternion Discrete Cosine Transform (QDCT) method in the phase spectrum method is firstly used to process the image to be located, so as to extract the saliency map of the image to be located.

[0105] It should be noted that since a single pixel is not sufficient to represent the feature information of the building facade, it is necessary to perform block processing on the building facade image.

[0106] In an embodiment of the present invention, a superpixel partitioning algorithm is used to partition a saliency map, and the saliency map is divided into salient areas and non-salient areas by thresholding the saliency map. Then, a grid is initialized for the salient areas and non-salient areas, and the initial superpixel center selects the grid center to search for pixels in the search area corresponding to each superpixel center, so that dense superpixels can be generated in salient areas and relatively sparse superpixels can be generated in non-salient areas, thereby reducing the number of global superpixels while retaining edge detail information, thereby finally generating a saliency partitioned map.

[0107] The acquired building area may be mixed with some non-building blocks, such as trees, grass, etc., and the area may not contain the complete building area. Therefore, it is necessary to further filter the significant block map.

[0108] Specifically, considering that the texture characteristics of buildings can be well preserved in superpixel blocks, the Euclidean distance between block feature vectors can be used as a similarity metric based on the texture characteristics of the building. The background blocks including the sky and the ground can be filtered out from the salient block map, and non-building elements can be excluded to screen out the building blocks. Then, the building blocks can be merged to obtain the initial outline of the building.

[0109] It should be noted that the image contour optimization using the level set method mainly represents the plane closed curve as the zero level set of the level set function, transforms the curve evolution problem in two-dimensional space into the surface evolution problem in three-dimensional space, and obtains the optimal contour by minimizing the defined energy function.

[0110] In this embodiment, by utilizing the level set method, the initial contour curve of the building is expressed in the form of an energy function. By minimizing the energy function, the curve is evolved toward the true contour, thereby optimizing the initial contour of the building, obtaining the true contour features of the building facade, and obtaining the building contour feature information.

[0111] The embodiment of the present invention introduces a visual saliency model based on image segmentation, takes the image saliency map as the basis, takes superpixel segmentation as the processing unit, and combines the level set evolution algorithm to achieve rapid and accurate extraction of building facade image contour features, improve the accuracy of building recognition, and help improve the accuracy of subsequent visual image positioning.

[0112] On the basis of any of the above embodiments, based on the pixel position information of the target contour point in the image to be located and the geographical location information of the target contour point, the posture information of the user terminal in the geodetic coordinate system is determined, including: based on the image to be located, determining the projection position information of each target contour point in the camera coordinate system.

[0113] Specifically, in step 103, the positioning device based on visual images extracts the 2D point projected on the image by the 3D point corresponding to each target contour point from the image to be positioned, and combines the two-dimensional pixel coordinates (i.e., 2D point coordinates) of each target contour point with the internal parameters calibrated in advance for the camera to restore the projection position information of each target contour point in the camera coordinate system.

[0114] Furthermore, based on the projection position information of each target contour point in the camera coordinate system and the geographical location information of each target contour point in the geodetic coordinate system, the position and posture information of the user terminal in the geodetic coordinate system is determined.

[0115] Specifically, the positioning device based on visual images adopts the PnP algorithm to solve the correspondence between the geographic location information (i.e., 3D coordinate values) of three or more target contour points in the geodetic coordinate system and their pixel location information (i.e., 2D projection coordinate values) in the camera coordinates corresponding to the image to be positioned, so as to determine the azimuth angle (Yaw), pitch angle (Pitch) and roll angle (Roll) of the user terminal when shooting the image to be positioned, and integrate the geographic location information of the user terminal in the geodetic coordinate system converted according to the relative position relationship between the user terminal and the captured content into the posture information output of the user terminal.

[0116] Among them, since there are three or more known point information in the image, there are multiple solutions to the linear equation established around the correspondence between the 3D coordinate value and its 2D projection coordinate value in the camera coordinate, so the unique solution can be determined by P3P, direct linear transformation, EPnP and other methods of estimating the pose of three pairs of points. In addition, the least squares problem can be constructed and solved iteratively in a nonlinear optimization manner, which is the common bundle adjustment (Bundle Adjustment, BA). The embodiment of the present invention does not specifically limit the solution method of the PnP algorithm.

[0117] The embodiment of the present invention substitutes the projection position information of the target contour point in the camera coordinate system and the absolute position information of the target contour point in the world coordinate system as matching point pairs into the PnP algorithm to solve the direction angle, pitch angle and roll angle of the user terminal itself. It can obtain the camera posture of the user terminal as an observer during observation through the geographic location information contained in the captured image, and use it as its own posture to initialize and calibrate the orientation, thereby improving the positioning and orientation accuracy of the equipment.

[0118] Figure 4 FIG. 1 is a schematic diagram of the structure of a positioning device based on visual images provided by the present invention. Figure 4 As shown, the positioning device based on visual images provided in the embodiment of the present invention includes: a surface feature extraction module 410, a coarse-grained positioning module 420 and a fine-grained positioning module 430, wherein:

[0119] The surface feature extraction module 410 is used to obtain surface feature information based on the image to be positioned collected by the user terminal when observing the area to be positioned.

[0120] The coarse-grained positioning module 420 is used to match the contour line corresponding to the surface feature information and the geographical location information of each contour point on the contour line from the surface morphology library.

[0121] The fine-grained positioning module 430 is used to determine the position and posture information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the image to be positioned and the geographical location information of the target contour point.

[0122] Among them, the target contour point is any contour point on the contour line; the surface morphology library is a spatial digital model of buildings and / or skylines constructed in advance based on terrain data obtained through ground observation by an aerial sensing device; the aerial sensing device includes a high-resolution satellite, and one or more of a camera or radar carried by a drone.

[0123] Specifically, the surface feature extraction module 410, the coarse-grained positioning module 420 and the fine-grained positioning module 430 are electrically connected in sequence.

[0124] The surface feature extraction module 410 uses the image captured by the user terminal of the scene in the area to be positioned as the image to be positioned, and uses an image processing algorithm to extract surface feature information for representing the topography from the image to be positioned.

[0125] The coarse-grained positioning module 420 uses the surface feature information obtained by the surface feature extraction module 410 as retrieval information to perform query matching in the surface morphology library, and the results are divided into two types: successful matching and failed matching.

[0126] A successful match means that the surface feature information contained in the current user's shooting picture can match the information stored in the locally stored surface morphology library, which means that the terrain features of the area have been included. Therefore, the contour line corresponding to the surface feature information obtained by shooting at a certain location in the current area can be extracted from the surface morphology library, and the geographical location information of each contour point on the contour line can be extracted from the spatial digital model.

[0127] Matching failure means that the surface feature information contained in the current user's shooting picture cannot match any information in the locally stored surface morphology library, which means that the aerial sensing device has not conducted any observation activities in the area. Therefore, it is necessary to first use the aerial sensing device to observe the current area and then construct the corresponding spatial digital model, and then carry out query and matching work on the surface feature information based on the obtained spatial digital model.

[0128] The fine-grained positioning module 430 extracts at least three or more contour points from the multiple contour points on the contour line as target contour points, applies them to the rear intersection principle, and calculates the position and posture information of the user terminal as the photographer in the geodetic coordinate system.

[0129] Optionally, the device further includes a first grid establishment module and a first index establishment module, wherein:

[0130] The first grid establishment module is used to establish a grid according to the degree of change of the terrain data obtained by the aerial sensing device from the ground observation.

[0131] The first index building module is used to use the skyline contour features calculated through the elevation data in each grid as the contour line index of the corresponding grid to generate the surface morphology library.

[0132] Optionally, the surface feature information includes skyline feature information;

[0133] Accordingly, the surface feature extraction module 410 includes a skyline detection unit and a skyline optimization unit, wherein:

[0134] The skyline detection unit is used to input the image to be located into a skyline detection model to obtain a skyline probability map output by the skyline detection model.

[0135] The skyline optimization unit is used to perform dynamic programming on the skyline probability graph and use the longest path formed by point-by-point connection as the skyline feature information.

[0136] The skyline detection model is trained based on sample remote sensing images and skyline labels annotated corresponding to the sample remote sensing images; the size of the skyline probability map is the same as the size of the image to be located, and the mapping value of each pixel point in the skyline probability map is the probability that the corresponding pixel point belongs to the skyline.

[0137] Optionally, the coarse-grained positioning module 420 is specifically used to, if it is determined that the area to be positioned is an urban environment, use the first skyline feature information as an index to query the first target grid from the surface morphology library, and then obtain the geographic location information of each contour point on the contour line based on the first target grid.

[0138] If it is determined that the area to be located is not an urban environment, the second skyline feature information and elevation data are used as indexes to query the second target grid from the surface morphology library, and then an equal-interval sampling serialization method is used within the second target grid to obtain the geographic location information of each contour point on the contour line.

[0139] The first skyline feature information is a panoramic skyline extracted from an image of a city scene taken vertically upward by a user terminal; and the second skyline feature information is a skyline extracted from an image of a non-city scene taken by a user terminal.

[0140] Optionally, the device further includes a second grid establishment module and a second index establishment module, wherein:

[0141] The second grid establishment module is used to establish a grid according to the degree of change of the terrain data obtained by the aerial sensing device from ground observation.

[0142] The second indexing module is used to extract the building outline features from the building facades in each grid using superpixel blocks as processing units as the contour line index of the corresponding grid to generate the surface morphology library.

[0143] Optionally, the fine-grained positioning module 430 includes a projection unit and a positioning unit, wherein:

[0144] The projection unit is used to determine the projection position information of each target contour point in the camera coordinate system based on the image to be positioned.

[0145] The positioning unit is used to determine the position information of the user terminal in the geodetic coordinate system based on the projection position information of each target contour point in the camera coordinate system and the geographical location information of each target contour point in the geodetic coordinate system.

[0146] The visual image-based positioning device provided in an embodiment of the present invention is used to execute the above-mentioned visual image-based positioning method of the present invention. Its implementation method is consistent with the implementation method of the visual image-based positioning method provided by the present invention, and can achieve the same beneficial effects, which will not be repeated here.

[0147] After the user terminal shoots the image to be located in the area to be located, the embodiment of the present invention uses the surface feature information extracted from the image to query the matching contour line and the geographical location area where it is located in the surface morphology library, and then uses the geographical location information and pixel location information of multiple target contour points in the geographical location area for fine positioning to obtain the position and posture of the user terminal when shooting the image to be located. It is possible to use aerospace remote sensing to quickly obtain surface images and morphological information when the area to be located is a satellite navigation denied area, and match this information with the ground image to achieve positioning and orientation for individuals, drones or light vehicles in the satellite navigation denied area while taking into account positioning efficiency and accuracy, which can avoid GNSS interference, magnetic field positioning requires the acquisition of relevant data in advance, radio positioning requires the construction of a large number of new infrastructure, and traditional visual positioning requires the construction of feature point clouds.

[0148] Figure 5 FIG. 1 is a schematic diagram of the structure of the positioning system based on visual images provided by the present invention. Figure 5 As shown, the visual image-based positioning system provided by the embodiment of the present invention includes an aviation sensing device 510, and a plurality of user terminals 520 communicatively connected to the aviation sensing device 510;

[0149] The aviation sensing device 510 is used to obtain terrain data by an aerospace method and transmit the terrain data to the user terminal 520 .

[0150] The user terminal 520 includes a camera 521 and a visual processor 522. The camera 521 observes the area to be positioned and sends the collected image to be positioned to the visual processor 522, so that the visual processor 522 can implement the positioning method based on visual images when executing the program.

[0151] Specifically, the positioning system based on visual images consists of an aerial sensing device 510 that can perform aerial measurements in the area to be positioned, and a user terminal 520 that can establish a communication connection with the aerial sensing device 510. Each user terminal 520 is carried by the user and is used to perceive the user's actual scene in the area to be positioned and then perform positioning and orientation work.

[0152] For example, Figure 6 FIG. 1 is a flow chart of a positioning system based on visual images provided by the present invention. Figure 6 As shown, the specific implementation process of the user terminal 520 and the aviation sensing device 510 performing the positioning and orientation work through the communication connection is as follows:

[0153] First, for the area to be located, terrain data is collected by an aerial sensing device 510 (such as a high-resolution satellite, drone or laser radar LiDAR, etc.) and transmitted to the visual processor 522 in the user terminal 520, so that the visual processor 522 can construct a high-precision DEM of the building and / or skyline in advance based on the terrain data, establish a grid according to a fixed or severe degree of terrain change, and construct a building or skyline index to form a surface morphology library.

[0154] When the user uses the camera 521 in the user terminal 520 to capture the scene in the area to be positioned, the image obtained is transmitted to the visual processor 522 as the image to be positioned, so that the visual processor 522 can use the image processing algorithm to extract the surface feature information for characterizing the topography from the image to be positioned.

[0155] The visual processor 522 uses the acquired surface feature information as retrieval information to perform rough grid matching in the surface morphology library. When the surface feature information contained in the current user's shooting picture matches the information stored in the locally stored surface morphology library, it means that the terrain features of the area have been included. Therefore, the contour line corresponding to the surface feature information obtained by shooting at a certain position in the current area can be extracted from the surface morphology library, and the geographical location information of each contour point on the contour line can be extracted from the spatial digital model.

[0156] The visual processor 522 refines the position and posture within the grid, that is, extracts at least three or more contour points from multiple contour points on the contour line as target contour points, applies them to the rear intersection principle, and calculates the posture information of the user terminal as the photographer in the geodetic coordinate system.

[0157] After the user terminal shoots the image to be located in the area to be located, the embodiment of the present invention uses the surface feature information extracted from the image to query the matching contour line and the geographical location area where it is located in the surface morphology library, and then uses the geographical location information and pixel location information of multiple target contour points in the geographical location area for fine positioning to obtain the position and posture of the user terminal when shooting the image to be located. It is possible to use aerospace remote sensing to quickly obtain surface images and morphological information when the area to be located is a satellite navigation denied area, and match this information with the ground image to achieve positioning and orientation for individuals, drones or light vehicles in the satellite navigation denied area while taking into account positioning efficiency and accuracy, which can avoid GNSS interference, magnetic field positioning requires the acquisition of relevant data in advance, radio positioning requires the construction of a large number of new infrastructure, and traditional visual positioning requires the construction of feature point clouds.

[0158] In addition, the above-mentioned logical instructions can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0159] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the visual image-based positioning method provided by the above-mentioned methods, which includes: obtaining surface feature information based on the image to be positioned collected by the user terminal through observation of the area to be positioned; matching the contour line corresponding to the surface feature information and the geographical location information of each contour point on the contour line from the surface morphology library; determining the posture information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the image to be positioned and the geographical location information of the target contour point; wherein the target contour point is any contour point on the contour line; the surface morphology library is a spatial digital model of buildings and / or skylines constructed in advance by terrain data obtained by ground observation by an aerial sensing device; the aerial sensing device includes a high-resolution satellite, and one or more of a camera or radar carried by a drone.

[0160] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the visual image-based positioning method provided by the above-mentioned methods, the method comprising: acquiring surface feature information based on the image to be positioned collected by the user terminal through observation of the area to be positioned; matching the contour line corresponding to the surface feature information and the geographical location information of each contour point on the contour line from the surface morphology library; determining the posture information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the image to be positioned and the geographical location information of the target contour point; wherein the target contour point is any contour point on the contour line; the surface morphology library is a spatial digital model of buildings and / or skylines constructed in advance by terrain data obtained by an aerial perception device through ground observation; the aerial perception device includes a high-resolution satellite, and one or more of a camera or radar carried by a drone.

[0161] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A positioning method based on visual images, characterized in that: include: Acquire surface feature information based on the image to be located collected by the user terminal through observation of the area to be located; Matching a contour line corresponding to the surface feature information and the geographical location information of each contour point on the contour line from a surface morphology library; Determine the position information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the image to be located and the geographical location information of the target contour point; Among them, the target contour point is any contour point on the contour line; the surface morphology library is a spatial digital model of buildings and / or skylines constructed in advance based on terrain data obtained through ground observation by an aerial sensing device; the aerial sensing device includes a high-resolution satellite, and one or more of a camera or radar carried by a drone.

2. The positioning method based on visual images according to claim 1, characterized in that: Before acquiring the surface feature information based on the image to be located collected by the user terminal through observation of the area to be located, the method further includes: Establishing a grid according to the degree of change of terrain data obtained by the aerial sensing device from earth observation; The skyline contour features calculated from the elevation data in each grid are used as the contour line index of the corresponding grid to generate the surface morphology library.

3. The positioning method based on visual images according to claim 2, characterized in that: The surface feature information includes skyline feature information; Accordingly, the step of acquiring the surface feature information based on the image to be located collected by the user terminal through observation of the area to be located includes: Inputting the image to be located into a skyline detection model to obtain a skyline probability map output by the skyline detection model; By performing dynamic programming on the skyline probability graph, the longest path formed by connecting points one by one is used as the skyline feature information; The skyline detection model is trained based on sample remote sensing images and skyline labels annotated corresponding to the sample remote sensing images.

4. The positioning method based on visual images according to claim 2, characterized in that: The contour line corresponding to the surface feature information matched from the surface morphology library, and the geographical location information of each contour point on the contour line, include: If it is determined that the area to be located is an urban environment, after querying a first target grid from the surface morphology library using the first skyline feature information as an index, the geographical location information of each contour point on the contour line is obtained based on the first target grid; If it is determined that the area to be located is not an urban environment, after querying the second target grid from the surface morphology library using the second skyline feature information and the elevation data as an index, an equal-interval sampling serialization method is adopted in the second target grid to obtain the geographical location information of each contour point on the contour line; The first skyline feature information is a panoramic skyline extracted from an image of a city scene taken vertically upward by a user terminal; and the second skyline feature information is a skyline extracted from an image of a non-city scene taken by a user terminal.

5. The positioning method based on visual images according to claim 1, characterized in that: Before acquiring the surface feature information based on the image to be located collected by the user terminal through observation of the area to be located, the method further includes: Establishing a grid according to the degree of change of terrain data obtained by the aerial sensing device from earth observation; In each grid, the building outline features extracted from the building facades are used as the contour line index of the corresponding grid to generate the surface morphology library with superpixel blocks as processing units.

6. The positioning method based on visual images according to claim 5, characterized in that: The surface feature information includes building outline feature information; Accordingly, the step of acquiring the surface feature information based on the image to be located collected by the user terminal through observation of the area to be located includes: Extracting a saliency map of the image to be located, and performing block processing on the saliency map using a superpixel block algorithm to generate a saliency block map; Extracting building blocks from the salient block map based on the texture features of the building, and merging the building blocks to obtain an initial outline of the building; The level set method is used to perform evolution calculation on the initial outline of the building to obtain building outline feature information.

7. The positioning method based on visual images according to any one of claims 1 to 6, characterized in that: The determining, based on the pixel position information of the target contour point in the image to be located and the geographical location information of the target contour point, the position and posture information of the user terminal in the geodetic coordinate system includes: Based on the image to be positioned, determining the projection position information of each target contour point in the camera coordinate system; Based on the projection position information of each target contour point in the camera coordinate system and the geographical location information of each target contour point in the geodetic coordinate system, the position and posture information of the user terminal in the geodetic coordinate system is determined.

8. A positioning device based on visual images, characterized in that: include: A surface feature extraction module is used to obtain surface feature information based on the image to be located collected by the user terminal when observing the area to be located; A coarse-grained positioning module is used to match the contour line corresponding to the surface feature information and the geographical location information of each contour point on the contour line from the surface morphology library; A fine-grained positioning module, used to determine the position and posture information of the user terminal in the geodetic coordinate system based on the pixel position information of the target contour point in the image to be positioned and the geographical location information of the target contour point; Among them, the target contour point is any contour point on the contour line; the surface morphology library is a spatial digital model of buildings and / or skylines constructed in advance based on terrain data obtained through ground observation by an aerial sensing device; the aerial sensing device includes a high-resolution satellite, and one or more of a camera or radar carried by a drone.

9. A positioning system based on visual images, characterized in that: It includes an aviation sensing device, and a plurality of user terminals that are communicatively connected to the aviation sensing device; The aviation sensing device is used to obtain terrain data by an aerospace method and transmit the terrain data to the user terminal; The user terminal includes a camera and a visual processor; the camera observes the area to be positioned and sends the collected image to be positioned to the visual processor, so that the visual processor executes the computer program to implement the positioning method based on visual images as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the visual image-based positioning method as described in any one of claims 1 to 7 is implemented.

Images