Positioning method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a positioning method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of image processing, in particular to the technical fields of image positioning, artificial intelligence and the like. According to the specific implementation scheme, an image to be positioned is obtained, image feature extraction is conducted on the image to be positioned, positioning image features corresponding to the image to be positioned are obtained, and positioning image feature vectors comprise low-dimensional image features and high-dimensional image features; matching the low-dimensional image features with a plurality of index features of an image position library to obtain at least one candidate index feature matched with the low-dimensional image features; the image position library stores position image features corresponding to a plurality of position images, index features constructed according to the position image features, and geographical position information associated with the position images; and in response to the condition that the candidate index feature does not meet a preset condition, matching the high-dimensional image feature with a position image feature corresponding to the candidate index feature, and determining geographical position information of the to-be-positioned image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to the fields of image localization and artificial intelligence. Specifically, this disclosure relates to a localization method and apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Positioning technology is a core supporting technology in intelligent fields such as autonomous driving, robot navigation, AR (augmented reality), VR (virtual reality), and drone operations. Its accuracy, environmental adaptability, and stability directly determine the functional limits of terminal products and user experience.

[0003] With the rapid development of artificial intelligence and mobile internet technology, the demand for location services in various industries has been upgraded from the traditional "approximate location judgment" to the stringent standards of "centimeter-level high precision, full-scene coverage, and strong anti-interference". Especially in complex scenarios such as indoor venues, underground parking lots, urban canyons, and tunnels, higher requirements are placed on the reliability of positioning technology. Summary of the Invention

[0004] This disclosure provides a positioning method and apparatus, an electronic device, and a computer-readable storage medium.

[0005] According to a first aspect of this disclosure, a positioning method is provided, the method comprising:

[0006] Obtain the image to be located, extract image features from the image to be located, and obtain the positioning image features corresponding to the image to be located. The positioning image feature vector includes low-dimensional image features and high-dimensional image features.

[0007] The low-dimensional image features are matched with multiple index features in the image location library to obtain at least one candidate index feature that matches the low-dimensional image features; the image location library stores the location image features corresponding to multiple location images, the index features constructed based on the location image features, and the geographical location information associated with the location images;

[0008] In response to the candidate index feature not meeting the preset conditions, the high-dimensional image feature is matched with the location image feature corresponding to the candidate index feature to determine the geographical location information of the image to be located.

[0009] According to a second aspect of this disclosure, a positioning device is provided, the device comprising:

[0010] The feature extraction module is used to acquire the image to be located, perform image feature extraction on the image to be located, and acquire the positioning image features corresponding to the image to be located. The positioning image feature vector includes low-dimensional image features and high-dimensional image features.

[0011] The preliminary matching module is used to match the low-dimensional image features with multiple index features in the image location library to obtain at least one candidate index feature that matches the low-dimensional image features; the image location library stores the location image features corresponding to multiple location images, the index features constructed based on the location image features, and the geographical location information associated with the location images;

[0012] The fine matching module is used to match the high-dimensional image features with the location image features corresponding to the candidate index features in response to the candidate index features not meeting the preset conditions, so as to determine the true geographical location information of the image to be located.

[0013] According to a third aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to at least one of the aforementioned processors; wherein,

[0016] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable at least one processor to perform the positioning method.

[0017] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described positioning method.

[0018] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described superposition method.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 This is a flowchart illustrating a positioning method provided in an embodiment of this disclosure;

[0022] Figure 2 This is a flowchart illustrating some steps of another positioning method provided in this embodiment of the present disclosure;

[0023] Figure 3This is a flowchart illustrating some steps of another positioning method provided in this embodiment of the present disclosure;

[0024] Figure 4 This is a flowchart illustrating some steps of another positioning method provided in this embodiment of the present disclosure;

[0025] Figure 5 This is a flowchart illustrating some steps of another positioning method provided in this embodiment of the present disclosure;

[0026] Figure 6 This is a flowchart illustrating some steps of another positioning method provided in this embodiment of the present disclosure;

[0027] Figure 7 This is a flowchart illustrating some steps of another positioning method provided in this embodiment of the present disclosure;

[0028] Figure 8 This is a schematic diagram of the structure of a positioning device provided in an embodiment of this disclosure;

[0029] Figure 9 This is a block diagram of an electronic device used to implement the positioning method of the embodiments of this disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] Currently, mainstream positioning technologies are mainly divided into two categories: positioning technologies based on global satellite navigation systems, such as GPS (Global Positioning System), and positioning technologies based on computer vision, such as SLAM (Simultaneous Localization and Mapping) technology based on visual features.

[0032] Among them, GPS-based positioning technology calculates the user's geographical location by receiving satellite signals. Due to its mature technology and low deployment cost, it has been widely used in civilian scenarios such as vehicle navigation and mobile phone maps.

[0033] However, it relies on external satellite signal transmission and has significant limitations: in indoor, underground, and dense building complexes where signals are blocked or shielded, signal loss or interruption is very likely to occur; even in open areas, due to the multipath effect (signal reflected by buildings and superimposed with direct signal), the positioning error in areas such as urban canyons can reach 10-30 meters, which cannot meet the high-precision requirements of autonomous driving obstacle avoidance and AR spatial anchoring.

[0034] In addition, GPS-based positioning technology requires a dedicated satellite receiving module, which is not friendly to cost-sensitive small devices (such as portable robots), and can only provide location coordinates, but cannot output scene semantic information, making it difficult to support the development of intelligent interactive functions.

[0035] Visual feature-based SLAM technology has become a research hotspot in the field of visual positioning in recent years. It captures environmental images through cameras or image sensors, extracts feature points, and constructs local maps to achieve real-time estimation of its own position and orientation. This technology does not rely on external satellite signals, showing a natural advantage in indoor and underground scenarios where GPS is unavailable, and can provide rich data such as object recognition and scene semantics through image analysis.

[0036] However, SLAM technology still has some insurmountable shortcomings: First, the cumulative error problem is prominent. As the positioning distance increases, the feature point matching deviation will continue to accumulate. Typically, the drift can reach 1-3 meters per 100 meters, and the accuracy will drop significantly after long-term operation. Second, it has weak resistance to dynamic interference. When there are moving targets such as pedestrians and vehicles in the environment, it is easy to cause feature point mismatch, which will lead to positioning jumps or failures. Third, SLAM mostly relies on real-time constructed local maps and lacks correlation with global geographic information, making it difficult to achieve large-scale positioning and map reuse across scenes.

[0037] The positioning methods and apparatuses, electronic devices, and computer-readable storage media provided in the embodiments of this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0038] The positioning method provided in this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0039] Figure 1 A flowchart illustrating the positioning method provided in an embodiment of this disclosure is shown. Figure 1 As shown in the figure, the positioning method provided in the embodiments of this disclosure may include steps S110, S120, and S130.

[0040] S110. Obtain the image to be located, extract image features from the image to be located, and obtain the positioning image features corresponding to the image to be located. The positioning image feature vector includes low-dimensional image features and high-dimensional image features.

[0041] The image to be located refers to the target image whose geographical location needs to be determined. It can be collected by terminal devices (such as cameras, AR glasses, and vehicle sensors) and contains visual information strongly correlated with the real scene. For example, the image to be located could be an image taken by a user indoors, such as in a shopping mall (including store logos, signs, etc.).

[0042] To extract image features from the image to be located, the corresponding location image features can be obtained by using a feature extraction model to extract image features from the image to be located, and the obtained feature vector is the location image feature.

[0043] The acquired localization image features can include both low-dimensional and high-dimensional image features. Low-dimensional image features can be feature vectors (typically 32-128 dimensions) obtained by dimensionality reduction and quantization of the original image features, i.e., the localization image features. These retain the core structural information of the image but discard some image details. High-dimensional image features can be high-dimensional feature vectors (typically 512-2048 dimensions) extracted from the localization image. These contain rich information such as image detail texture and semantic attributes (e.g., object category, scene structure).

[0044] In some possible implementations, low-dimensional image features of the localization image can be extracted using a lightweight network, while high-dimensional image features of the localization image can be extracted using a deep network.

[0045] In some possible implementations, high-dimensional image features of the location image can be extracted through a network, and then the high-dimensional image features can be reduced and quantized to obtain low-dimensional image features of the location image.

[0046] S120. Match the low-dimensional image features with multiple index features of the image location library to obtain at least one candidate index feature that matches the low-dimensional image features.

[0047] The image location library stores the location image features corresponding to multiple location images, the index features constructed based on the location image features, and the geographical location information associated with the location images.

[0048] The image location database can be a structured database that stores massive amounts of "location-image-feature" related data, and it mainly contains the following information:

[0049] Location images: Real-world images covering the target area (such as street views, indoor scenes, and multi-view images of landmark buildings);

[0050] Location image features: High-dimensional image features corresponding to each location image, which can be high-dimensional feature vectors obtained by inputting the location image into a deep network for feature extraction;

[0051] Index features: Low-dimensional indexes are constructed based on the location image features of the location image. These can be feature vectors or quantized codes obtained by using quantization techniques such as product quantization, scalar quantization, principal component analysis, etc. to reduce the dimensionality of the location image features.

[0052] Geographic location information: precise coordinates associated with the location image (such as latitude and longitude, indoor coordinates) and scene labels (such as "XX shopping mall XX floor", "XX intersection", etc.).

[0053] For example, street view images with precise geographic coordinates can be obtained from street view map data as location images, along with their corresponding geographic location information. Alternatively, location images can be filtered from user-uploaded photos with geotags. Feature extraction is then performed on the filtered location images to obtain location image features and index features. The resulting image location library can be a database containing a large number of location images, where each location image's location image feature is a 2048-dimensional high-dimensional feature vector, and its index feature is a 64-dimensional low-dimensional feature vector.

[0054] In some possible implementations, after obtaining the low-dimensional and high-dimensional image features of the image to be located, the low-dimensional image features are used to perform a "preliminary match" with multiple index features in the image location database. Specifically, the similarity between the low-dimensional image features and all index features in the image location database can be calculated, including the top N index features with the highest similarity as candidate index features.

[0055] S130. In response to the candidate index features not meeting the preset conditions, the high-dimensional image features are matched with the location image features corresponding to the candidate index features to determine the real geographical location information of the image to be located.

[0056] The preset conditions can be that the similarity between the candidate index features and the low-dimensional image features is greater than a preset threshold, or that the number of candidate index features is less than a preset number.

[0057] In other words, when the similarity between the candidate index features and the low-dimensional image features is small (less than a preset threshold), or when the number of candidate index features is large (greater than a preset number), the high-dimensional image features of the image to be located are obtained (e.g., a request is sent to upload high-dimensional image features) and "fine matching" is performed using the high-dimensional image features.

[0058] Among them, "fine matching" can be to retrieve the high-dimensional feature vectors corresponding to the candidate index features, that is, the location image features, calculate the precise similarity between the high-dimensional image features of the image to be located and these high-dimensional feature vectors one by one, select the location image with the highest similarity, and determine the associated geographical location information of the image to be located as the real geographical location information of the image to be located.

[0059] Since the similarity calculation of image features (such as cosine similarity and Euclidean distance) is essentially a "multiplication and addition operation between vectors", the amount of computation is positively correlated with the feature dimension. For example, the similarity calculation of 1024-dimensional high-dimensional features requires 1024 multiplications + 1023 additions; while 64-dimensional low-dimensional features only require 64 multiplications + 63 additions. Therefore, using low-dimensional image features for "preliminary matching" can quickly narrow down the range of "fine matching". Compared with directly using image features for matching, it is equivalent to using low-dimensional feature matching instead of high-dimensional feature matching, which can greatly reduce the amount of computation and improve the matching speed.

[0060] At the same time, the process of acquiring low-dimensional image features does not involve randomly discarding information, but rather retaining the image's "most distinctive core structural features" (such as "traffic lights at intersections" or "escalators indoors") and discarding secondary information such as "texture details and lighting changes." This can fully meet the large-scale screening needs of "preliminary matching."

[0061] In the specific matching process, low-dimensional features can be directly adapted to lightweight retrieval technologies such as "quantized index" and "hash index". After converting low-dimensional features into "binary code" or "quantized code", complex vector calculations can be replaced by "table lookup comparison", which can further accelerate the speed of "preliminary matching".

[0062] The higher the feature dimension, the richer the image information it can carry. It not only includes "macro structure" but also covers key difference information such as "detail texture, semantic attributes, and environmental associations"—these are the core of distinguishing "similar but different real scenes." Therefore, "fine matching" using high-dimensional image features can distinguish details that "preliminary matching" cannot distinguish, thereby improving the accuracy of positioning.

[0063] For example, when dealing with intersections with similar layouts, low-dimensional image features may be misidentified as the same location due to the "same intersection layout"; however, high-dimensional image features can capture the subtle differences in the details of the walls at the intersection in the image, achieving accurate differentiation.

[0064] The hierarchical matching of low-dimensional and high-dimensional image features is essentially "allocating computing resources on demand". High-dimensional fine matching is triggered only for scenarios where low-dimensional initial screening is "unreliable" (such as low confidence and large number of candidate index features). This avoids both "wasting resources of full high-dimensional computation" and "insufficient accuracy of single low-dimensional matching".

[0065] In the positioning method provided in this disclosure, feature extraction is performed on the image, and then feature matching technology is used to achieve accurate matching between the image to be located and the image features in the image location database, thereby achieving accurate positioning. Since it does not rely on external devices (such as GPS) and can locate solely through image features, accurate positioning can still be achieved in indoor and underground scenes where GPS is unavailable. Because image features are more resistant to interference from lighting and seasonal changes, they are more robust and adaptable to different scenes. Simultaneously, by using low-dimensional image features to quickly narrow down the matching range, and high-dimensional image features to ensure accurate hierarchical matching, it avoids both the "waste of resources from full high-dimensional computation" and the "insufficient accuracy of single low-dimensional matching."

[0066] The positioning method provided in the embodiments of this disclosure will be described in detail below.

[0067] In some possible implementations, if the candidate index features obtained in step S120 meet preset conditions, such as high confidence of the candidate index features and a large number of candidate index features, then "fine matching" can be discarded.

[0068] Figure 2 The diagram illustrates a process for determining the true geographic location information of an image to be located directly based on the "preliminary matching" result when candidate index features meet preset conditions. Figure 2 As shown, steps S210 and S220 may be included.

[0069] S210. In response to the candidate index features meeting the preset conditions, determine at least one candidate geographic location information corresponding to the image to be located based on the geographic location information associated with the location image corresponding to the candidate index features.

[0070] The preset conditions can be that the similarity between the candidate index features and the low-dimensional image features is greater than a preset threshold, or that the number of candidate index features is less than a preset number.

[0071] In other words, when the similarity between the candidate index features and the low-dimensional image features is large (greater than the preset threshold), or when the number of candidate index features is small (less than the preset number), the geographical location information corresponding to the candidate index features is directly used as the candidate geographical location information.

[0072] S220. Determine the true geographical location information of the image to be located based on at least one candidate geographical location information.

[0073] In some possible implementations, cluster analysis can be performed on all latitude and longitude coordinates in the candidate geographic location information. Since these coordinate points typically originate from multiple images near the same location, they will spatially cluster into one or a few small regions. The system selects the cluster region with the most data points and uses the geographic location of the cluster center (e.g., calculating the average latitude and longitude of all points in the cluster) as the final true geographic location, and the corresponding information is the true geographic location information. This method can effectively eliminate the interference of outliers, such as an occasionally matched image that is distant but coincidentally similar in features, thus greatly improving the reliability and stability of the results.

[0074] In some possible implementations, the calculated similarity score can be used as a weight. The latitude and longitude of each candidate geographic location are multiplied by the similarity score (normalized) of its corresponding image. Then, the average of all weighted coordinate values ​​is calculated to obtain the final weighted average geographic location, which is the true geographic location information. Through weighted averaging, images with higher similarity have greater "weight" in the final positioning result, making the final result more inclined to favor the locations of the most similar images. This smooths out multiple results and further improves positioning accuracy.

[0075] In some possible implementations, other auxiliary information can be used to determine the true geographic location information of the image to be located from the candidate geographic location information.

[0076] In scenarios such as map POI (Point of Interest) production and autonomous driving, image location databases typically contain hundreds of billions of image data. If high-dimensional features are used for full database matching, a single search can take several seconds or even minutes, which cannot meet the "real-time positioning" requirement. However, the initial screening of low-dimensional features can compress the candidate set from "hundreds of billions" to "thousands" in a very short time. Directly using the matching results of low-dimensional image features can compress the positioning response speed to "milliseconds", meeting the "real-time positioning requirement".

[0077] Meanwhile, low-dimensional image feature matching consumes less computing power and storage. This lightweight characteristic makes it suitable for resource-limited devices such as "vehicle terminals and AR glasses". "Fine matching" is only performed when the similarity between the candidate index feature and the low-dimensional image feature is small (less than the preset threshold) or when the number of candidate index features is large (greater than the preset number). This allows these resource-limited devices to handle most positioning situations. Only a few positioning situations require cloud processing of high-dimensional image features to obtain positioning results.

[0078] The ultimate goal of positioning is to "provide reliable location information for terminal applications," not to "pursue the completeness of technical steps." When "preliminary matching" has already given a result with "high confidence and consistency among multiple candidates," the "additional accuracy improvement" of "fine matching" (possibly reducing the error from "1 meter error" to "0.5 meter error") is "redundant accuracy" for most applications—users or devices cannot perceive this subtle difference at all, but it comes at the cost of "response delay and device lag," which is unnecessary. By discarding "fine matching" in steps S210 and S220, this cost can be avoided.

[0079] When "preliminary matching" fails to provide results with "high confidence and consistency among multiple candidates," the "additional accuracy improvement" of "refined matching" can bring about a significant increase in accuracy. Therefore, if the candidate index features obtained in step S210 do not meet the preset conditions, such as having low similarity to low-dimensional image features (less than a preset threshold), or having a large number of candidate index features (greater than a preset number), it is necessary to obtain high-dimensional image features of the image to be located (e.g., send a request to upload high-dimensional image features) and use the high-dimensional image features for "refined matching."

[0080] Figure 3 The diagram illustrates a flowchart of one implementation of "fine-grained matching" using high-dimensional image features, such as... Figure 3 As shown, steps S310, S320, and S330 may be included.

[0081] S310. Match the high-dimensional image features with the location image features corresponding to the candidate index features to obtain at least one location image feature that matches the high-dimensional image features.

[0082] Specifically, the distance or similarity between the high-dimensional image features of the image to be located and the high-dimensional feature vector of each candidate location image is calculated. Common methods include calculating Euclidean distance, cosine similarity, or inner product. The result is a quantified similarity score; a higher score (or a smaller distance) indicates greater similarity between the two images.

[0083] Based on the calculated similarity scores, all results are sorted (from high to low or from low to high), and the top K results are selected. The location image features corresponding to these K results are the "target location image features" most similar to the image to be located. The value of K can be adjusted according to the actual application scenario. For example, in scenarios requiring high accuracy, K = 1 or 3 can be used; in complex scenarios, K = 10 or more can be used to provide sufficient data samples for subsequent processing.

[0084] S320. Determine at least one candidate geographic location information corresponding to the image to be located based on the geographic location information associated with the location image features.

[0085] Each image feature in the image location database is strictly bound to the geographic location information (such as latitude and longitude coordinates) of its shooting location. By querying the above K "target location image features", the latitude and longitude coordinates corresponding to each feature are obtained. These coordinate points together form a "candidate geographic location set". This set reflects several possible locations where the image to be located may be.

[0086] S330. Determine the true geographical location information of the image to be located based on at least one candidate geographical location information.

[0087] In some possible implementations, cluster analysis can be performed on all latitude and longitude coordinates in the candidate geographic location set. Since these coordinate points typically originate from multiple images near the same location, they spatially cluster into one or a few small regions. The system selects the cluster region with the most data points and uses the geographic location of the cluster center (e.g., calculating the average latitude and longitude of all points in the cluster) as the final true geographic location. This information is the true geographic location information. This method effectively eliminates the interference of outliers, such as an occasionally matched image that is distant but coincidentally similar in features, thus greatly improving the reliability and stability of the results.

[0088] In some possible implementations, the calculated similarity score can be used as a weight. The latitude and longitude of each candidate geographic location are multiplied by the similarity score (normalized) of its corresponding image. Then, the average of all weighted coordinate values ​​is calculated to obtain the final weighted average geographic location, which is the true geographic location information. Through weighted averaging, images with higher similarity have greater "weight" in the final positioning result, making the final result more inclined to favor the locations of the most similar images. This smooths out multiple results and further improves positioning accuracy.

[0089] In some possible implementations, other auxiliary information can be used to determine the true geographic location information of the image to be located from the candidate geographic location information.

[0090] Compared to low-dimensional image features, which are prone to misjudgment due to loss of details and interference from factors such as changes in lighting, dynamic objects, and seasonal changes, high-dimensional image features can capture "unchanging semantic cores" (such as road sign text and building outlines) and resist environmental interference. Therefore, high-dimensional image feature matching can help obtain more accurate localization results.

[0091] For example, at the same tunnel entrance, low-dimensional image features may be misjudged as different locations due to the difference in brightness between "strong daylight" and "nighttime lighting"; however, high-dimensional features can focus on invariant semantics such as "the arched structure of the tunnel entrance and the speed limit sign text next to it", achieving accurate matching across time periods.

[0092] pass Figure 2 and Figure 3 The implementation shown forms a hierarchical matching and positioning structure. By using low-dimensional image features to quickly narrow down the matching range, and high-dimensional image features to ensure accurate hierarchical matching, it avoids both the "waste of resources in full high-dimensional computation" and the "insufficient accuracy of single low-dimensional matching".

[0093] In some possible implementations, steps S220 and S330 can determine the true geographic location information of the image to be located based on at least one candidate geographic location information, which may be done by using semantic information to determine the true geographic location information of the image to be located from the candidate geographic location information.

[0094] Specifically, semantic verification is performed on at least one candidate geographic location information, and the true geographic location information of the image to be located is determined based on the semantic verification results.

[0095] Semantic verification can include multimodal semantic verification, scene semantic verification, and shooting time verification.

[0096] The multimodal semantic verification process may include: extracting text information from the image to be located, calculating the semantic similarity between the text information and candidate geographic location information, and filtering candidate geographic location information based on semantic similarity.

[0097] Specifically, an OCR (Optical Character Recognition) model can be used to extract text from the image to be located, obtaining text information. Alternatively, an image description generation model can be used to generate a text description of the image to be located, obtaining text information. The semantic similarity between the text information and the text description information in the candidate geographic location information can then be calculated, and the candidate geographic location information can be reordered or filtered based on the semantic similarity.

[0098] The process of scene semantic verification may include: identifying the image scene corresponding to the image to be located; and filtering the candidate geographic location information based on the consistency between the image scene and the image scene corresponding to the candidate geographic location information.

[0099] Specifically, the scene recognition model is used to identify the image scene corresponding to the image to be located, and the scene label corresponding to the image scene of the image to be located is obtained. The scene label in the candidate geographic location information is obtained as the image scene corresponding to the candidate geographic location information. If the scene labels are inconsistent, the candidate geographic location information is retained; if they are inconsistent, the confidence of the candidate geographic location information is reduced or it is filtered out from the candidate results.

[0100] The process of verifying the shooting time may include: obtaining the shooting time information of the image to be located; determining whether the shooting time is within the business hours range of the geographical location corresponding to the candidate geographical location information; if it is not within the business hours, reducing the confidence of the candidate geographical location information or filtering it out from the candidate results.

[0101] In some possible implementations, the semantic verification result can be used as a weighted factor and fused with the similarity score of "fine matching" to obtain a comprehensive confidence score; the final real geographical location information can then be selected based on the comprehensive confidence score.

[0102] In the positioning method provided in the embodiments of this disclosure, the construction of an image location database is the basis for matching. Therefore, the positioning method provided in the embodiments of this disclosure may also include the process of constructing an image location database.

[0103] Figure 4 A flowchart illustrating one implementation of building an image location library is shown, such as... Figure 4 As shown, steps S410, S420, and S430 may be included.

[0104] S410. Obtain multiple location images and the geographic location information associated with the location images.

[0105] Among them, the location image is a real-world image covering the target location area. It is acquired by professional acquisition equipment (such as street view acquisition vehicles, indoor scanners, and drones) according to a preset sampling strategy. It includes scene data from multiple perspectives (horizontal, elevation, and depression) and multiple time periods (weekdays / weekends, daytime / nighttimes) to cover environmental changes.

[0106] The associated geographic location information is obtained through high-precision positioning equipment. Indoor scenes are accurate to the "floor + plane coordinates" (e.g., "3rd floor of XX shopping mall, X=15.2m, Y=8.7m"), while outdoor scenes are accurate to the latitude and longitude at the centimeter level (e.g., "116.3245°E, 39.9123°N").

[0107] S420. Extract image features from the location image, obtain the location image features corresponding to the location image, and construct index features based on the location image features.

[0108] Specifically, high-dimensional image features, or location image features, are extracted from each location image using a deep neural network. These features retain rich information such as detailed textures and semantic attributes (e.g., object category, scene structure). Low-dimensional index features are then constructed by performing dimensionality reduction and quantization processing on these location image features (e.g., dimensionality reduction through principal component analysis followed by product quantization to generate quantized codes).

[0109] S430. Link and store the geographic location information associated with the location image, the location image features corresponding to the location image, and the index features corresponding to the location image to construct an image location library.

[0110] Among some possible implementations, a distributed database can be used to implement relational storage:

[0111] Using the unique identifier of the location image (such as acquisition time + device ID) as the key, the associated storage fields include: the original location image, location image features, index features, geographic location information (such as latitude and longitude / indoor coordinates, scene label), and acquisition timestamp.

[0112] This structured storage ensures precise binding of feature vectors to geographic locations, providing a data foundation for subsequent matching.

[0113] In some possible implementations, after acquiring the location image, the location image can be preprocessed and enhanced to increase the number of location images in the image location library.

[0114] Specifically, location images can be standardized and scaled to facilitate image feature extraction; random cropping and flipping can be applied to introduce angle transformations; color dithering or blurring can be applied to introduce noise transformations; and brightness transformations can be applied to simulate weather and lighting changes to introduce lighting transformations, increasing the diversity of location images so that the feature extraction model can obtain more general features from location images.

[0115] In some possible implementations, image feature extraction of the location image to obtain the location image features can specifically include:

[0116] A deep convolutional neural network optimized with a metric learning loss function is used to extract image features from the location image to obtain the location image features corresponding to the location image.

[0117] Specifically, a CNN (Convolutional Neural Network) model optimized with metric learning loss functions such as Triplet Loss and Contrastive Loss is used to extract image features from location images. This results in features that are "close in similar scenes and far in dissimilar scenes." For example, after optimization, the feature similarity of "images of the same convenience store from different angles" is significantly higher than that of "images of adjacent convenience stores."

[0118] The visual transformer model is used to extract image features from the location image to obtain the location image features corresponding to the location image.

[0119] Specifically, the visual transformer model segments images into a sequence of patches and captures global semantic relationships (such as "the spatial relationship between escalators and shops in a shopping mall") through a self-attention mechanism. It can extract features of complex scenes in location images and outperforms traditional CNNs in large indoor scenes.

[0120] The location image features are extracted by using a visual model pre-trained based on self-supervised learning.

[0121] Specifically, a visual model pre-trained with unlabeled data is used to learn general visual features of location images through tasks such as "contrastive learning" and "mask reconstruction". This method does not rely on large-scale labeled data and can still maintain feature robustness in niche scenarios (such as industrial workshops), making it suitable for feature extraction of location images in niche scenarios.

[0122] A multimodal contrastive learning model is used to extract image features from the location image to obtain the location image features corresponding to the location image.

[0123] Multimodal contrastive learning models such as CLIP (Contrastive Language–Image Pre-training) are used to align image features with text semantics (such as “supermarket cashier”, “hospital registration office”). The extracted location image features not only contain visual information but also implicit semantic labels, improving the reliability of cross-modal matching.

[0124] In some possible implementations, image features can be extracted from the location image using one or more of the methods described above.

[0125] In some possible implementations, the location image features corresponding to the acquired location image can be normalized; the normalized location image features can then be quantized to construct the index features.

[0126] Specifically, the location image features obtained through one or more methods are normalized, such as L2 normalization, to scale the obtained location image features to a unit sphere, eliminating feature scale deviations caused by differences in brightness and contrast between different images, and avoiding scale differences from affecting subsequent similarity calculations.

[0127] The normalized high-dimensional features are reduced in dimensionality and quantized. Specifically, principal component analysis is used to retain most of the information while compressing the dimensionality of the location image features. Product quantization is used to divide the compressed features into multiple segments, each segment is represented by a fixed number of bits of binary code, and finally a fixed-byte quantized code is generated as the index feature.

[0128] The quantized index features have lower storage costs and support fast "table lookup" matching, which greatly improves matching efficiency.

[0129] In some possible implementations, matching efficiency can also be improved by building an index structure.

[0130] Figure 5 The diagram illustrates a flowchart of one implementation of constructing an index structure, such as... Figure 5 As shown, step S510 may be included.

[0131] S510. Construct a first index structure based on the index features corresponding to the location image; construct a second index structure based on the location image features corresponding to the location image.

[0132] The first index structure, which is constructed based on the index features of all location images, can be constructed by using lightweight indexing technology to build a mapping relationship of "quantization code-index feature-location image ID", or it can be constructed by using a clustering algorithm.

[0133] Specifically, a clustering algorithm suitable for large-scale feature data is used to cluster the index features of all location images: the clustering algorithm is used to cluster the index features corresponding to all location images to form multiple cluster centers; each index feature is assigned to its respective cluster center to form the first index structure.

[0134] Taking the K-means clustering method as an example, the number of clusters K is preset, and K cluster centers are found through iterative calculation. Each center represents a "set of index features with similar characteristics". For example, index features related to "shopping mall escalator entrance" are clustered into one class, and the cluster center vector reflects the core features of that class.

[0135] Calculate the distance (e.g., Euclidean distance) between each index feature and the K cluster centers, and assign it to the nearest cluster center to form a hierarchical structure of "cluster center - sub-cluster - index feature".

[0136] This structure eliminates the need to traverse the entire database during "initial matching." Instead, it only requires matching cluster centers first and then searching within sub-clusters, improving retrieval efficiency by a factor of K.

[0137] A second index structure is constructed based on the location image features of all location images. High-precision indexing techniques (such as image indexing and ball tree) are used to form a topological structure of "feature vector - adjacent vector - location image ID".

[0138] Specifically, the second index structure can be a graph-based nearest neighbor index structure.

[0139] The core of the graph-based nearest neighbor index structure is to use location image features as "nodes" and connect nodes with feature similarity higher than a threshold using "edges" to form a topological network. When constructing the second index structure, for each location image feature, its K most similar nearest neighbor vectors are searched, and bidirectional connections are established (if A's nearest neighbor includes B, then B's nearest neighbor also includes A). For example, high-dimensional features of "location images taken from different angles at the same intersection" will be interconnected to form a dense subgraph.

[0140] Specifically, the second index structure is formed by iteratively inserting each node and constructing a multi-level graph structure; wherein, the upper-level graph is randomly sampled from the lower-level graph and has a lower density than the lower-level graph, and the bottom-level graph contains all nodes; in each level graph, multiple nearest neighbor entry nodes are found for the newly inserted node and connections are established with the entry nodes, so that similar feature vectors are connected to each other in the graph.

[0141] During the matching process, starting from any initial node, the system gradually approaches the target feature (the high-dimensional image features of the image to be located) by traversing the nearest neighbor nodes, eventually finding the node with the highest similarity. Compared to brute-force search, graph indexing reduces the time complexity of high-dimensional feature matching from O(N) to O(logN), making it particularly suitable for fast retrieval of high-dimensional features exceeding one million.

[0142] Figure 6 The diagram illustrates a flowchart of an implementation method that uses the index structures for matching after constructing the first and second index structures. Figure 6 As shown, steps S610 and S620 may be included.

[0143] S610. Search for candidate index features that are similar to the low-dimensional image features in the first index structure.

[0144] When using low-dimensional image features (low-dimensional image features of the image to be located) for matching, the search is performed directly in the first index structure: the cluster center is matched first by quantization code, and after the cluster center is matched, the search is performed within the sub-cluster to locate similar index features, and candidate index features are selected to achieve compression from "hundreds of billions" to "thousands" or even larger ranges.

[0145] S620. In the second index structure, based on high-dimensional image features, the positional image features corresponding to the candidate index features are searched.

[0146] When "fine matching" (matching using high-dimensional image features of the image to be located) is required, the search is performed in the second index structure: based on the location image ID associated with the candidate index features, the corresponding location image features are retrieved, and the feature vector most similar to the high-dimensional image features of the image to be located is quickly found in the second index structure through topological relationships, achieving precise filtering "from thousands to single digits".

[0147] In some possible implementations, after building the image location library, newly added location images are captured in real time and the image location library is updated accordingly.

[0148] In some possible implementations, the process involves monitoring and acquiring newly added location images and their associated geographic location information; extracting image features from the newly added location images to obtain corresponding location image features, and constructing index features based on these features; adding the associated geographic location information, the corresponding location image features, and the corresponding index features to the image library; incrementally updating the first index structure based on the corresponding index features of the newly added location images; and incrementally updating the second index structure based on the corresponding location image features of the newly added location images.

[0149] Specifically, newly added location images (such as user-uploaded POI real-world images and periodically collected new scene images) are captured in real time, and their associated geographic location information (such as user-reported coordinates and location data from the acquisition device) is obtained simultaneously. Feature extraction is performed on the newly added images: location image features and index features are generated. The specific process is the same as that in the initial construction and will not be repeated here.

[0150] New data (geographic location, location image features, index features) is associated and stored in the database to avoid rebuilding the entire database.

[0151] Calculate the distance between the newly added index features and the existing cluster centers, and assign them to the corresponding clusters; if the distance exceeds the threshold, add new cluster centers to avoid cluster imbalance and update the first index structure.

[0152] For newly added location image features, search for nearest neighbor nodes and establish connections, while updating the nearest neighbor list of existing nodes (if the newly added features are more similar to existing nodes), and updating the second index structure.

[0153] This incremental update mechanism avoids the high cost of "full index reconstruction" and ensures that the image library can maintain its positioning accuracy when the city is built or the scene changes (such as the addition of shops or road renovation).

[0154] In some possible implementations, a hotspot index can also be implemented by constructing a hotspot data index based on the features corresponding to frequently accessed geographical locations.

[0155] Figure 7 The diagram illustrates a flowchart of one implementation method for constructing a hotspot data index and using the hotspot data index for matching, as shown below. Figure 7 As shown, steps S710, S720, and S730 may be included.

[0156] S710: Analyze the geographical locations of high-frequency visits based on historical data.

[0157] Based on historical location request data, geographical locations that exceed a threshold (e.g., 1000 times) within a unit of time (e.g., 24 hours) are marked as "hotspot areas" (e.g., popular business districts, transportation hubs, office building lobbies), i.e., frequently accessed geographical locations.

[0158] S720. Construct a hotspot data index based on the location image features corresponding to the location images of the frequently accessed geographical locations.

[0159] Extract location image features from all location images of the "hotspot area", construct an independent hotspot data index, and store it in a cache or high-performance in-memory database.

[0160] S730. Match the location image features with the location image features in the hotspot data index.

[0161] After obtaining the high-dimensional image features of the image to be located, the high-dimensional image features are used to match the hotspot data index first. If the match is successful, the result is returned directly; otherwise, the regular "preliminary matching" process is entered.

[0162] Location requests for hotspot areas (such as popular business districts, transportation hubs, and office building lobbies) typically account for 30%-60% of total requests. Relying on conventional indexing processes (i.e., "preliminary matching" -> "fine matching") takes a long time for a single location attempt. Hotspot data indexing, however, stores the location image features of these areas in a high-performance in-memory database or cache, and builds a dedicated lightweight index. This allows for skipping the full database scan, achieving a simplified process of "location request -> hotspot index matching -> result return," thus reducing response time.

[0163] In some possible implementations, hotspot indexes can be deployed on separate high-performance servers, isolated from the computing resources of regular indexes; location requests in non-hotspot areas still follow the regular process, avoiding resource "crowding" by requests from hotspot points.

[0164] The application of the positioning method provided in this disclosure will be specifically described below with reference to specific embodiments.

[0165] Example 1: Construction of an image location library

[0166] Data acquisition and preprocessing configuration: The acquisition equipment is a street view acquisition vehicle equipped with a 4K camera, covering the core areas of multiple cities. The images include multiple time periods and multiple weather scenes; the preset standardized scaling size is configured with random cropping, horizontal flipping, color dithering, Gaussian blur and weather simulation algorithms.

[0167] Model cluster deployment: Deploy multi-model backbone networks according to scene type. Outdoor street scene images prioritize the use of multimodal contrastive learning models, tunnel / underground parking lot images prioritize the use of self-supervised learning pre-trained visual models, and indoor shopping mall images prioritize the use of deep convolutional neural networks optimized by metric learning loss functions and visual transformer models. All model output vectors have the same dimension.

[0168] The street view acquisition vehicle uploads newly added street view images to the cloud, with each image accompanied by metadata (acquisition time, geographical location, scene tags, etc.).

[0169] Data preprocessing:

[0170] Standardized scaling: Scaling images uniformly to a preset pixel size to eliminate resolution differences;

[0171] Random Enhancement: Triggers random horizontal flipping and color jitter to simulate visual changes under different lighting conditions, and additionally overlays lighting filters to supplement the characteristics of different weather scenes and improve environmental adaptability.

[0172] The system automatically matches a suitable model based on the scene labeling system, and uses the model to extract image features from the street view image. The original feature vector output by the model is normalized, and the vector magnitude is compressed to 1.0 to eliminate feature scale deviations caused by shooting distance and light intensity in different images, ensuring uniform vector distribution.

[0173] Generate standardized dense feature vectors, attach images to the vectors, store them in an image location library, and bind them to the coordinates of the image.

[0174] Example 2: Positioning Example

[0175] I. Implementation Scenarios

[0176] When users wear devices, such as AR glasses, to capture images inside a parking lot, they need to locate the specific parking space area within a short period of time.

[0177] II. Preparatory Work

[0178] Image location library and index construction: An image location library for the target parking lot has been built;

[0179] The indexing system is deployed in a hierarchical structure: the first layer is the product quantization index (compact features), the second layer is the graph index (high-dimensional features), and hotspot area features (elevator entrance, payment machine) are stored in multi-level caches.

[0180] Confidence and verification rule configuration: Set confidence thresholds and integrate OCR text recognition and cross-modal retrieval.

[0181] III. Implementation Steps

[0182] 1. Input the image the user is querying.

[0183] AR glasses capture images of an underground parking lot, including pillars, license plates of adjacent parking spaces, and green elevator signs on the walls, which are then uploaded to a cloud-based location service via a 5G network.

[0184] 2. Upload Image Feature Extraction

[0185] The cloud-based system calls a model adapted to low-light underground scenarios to extract features from uploaded images, generating high-dimensional feature vectors. At the same time, it uses PCA to reduce the dimensionality and generate compact feature vectors.

[0186] 3. Cloud-based reception of feature vectors and confidence level judgment

[0187] The cloud prioritizes receiving compact feature vectors and performs coarse matching with the first-layer quantization index to calculate candidate index features. If all of these are below the confidence threshold, they are judged as "low confidence". The backend system sends a request to the AR glasses to obtain high-dimensional feature vectors.

[0188] 4. Hierarchical Hybrid Index Retrieval

[0189] The first layer of coarse search: Based on geospatial priors, pruning and excluding index data from other regions, and quickly filtering out multiple candidate index features through quantified indexes;

[0190] The second layer of fine retrieval involves finely sorting the high-dimensional image features corresponding to the candidate index features in the graph index, calculating the high-dimensional feature similarity, and obtaining multiple candidate results.

[0191] 5. Multimodal / Semantic Validation

[0192] Text extraction: OCR recognition of text in query images, such as pillars or license plates;

[0193] Cross-modal verification: The multimodal model is called to calculate the semantic similarity between the text associated with the candidate results and the extracted text, determine the final result, and use the geographical location information corresponding to the result as the real geographical location information of the uploaded image.

[0194] Based on and Figure 1 The method shown follows the same principle. Figure 8 A schematic diagram of the structure of a positioning device provided in an embodiment of this disclosure is shown, such as... Figure 8 As shown, the positioning device 80 may include:

[0195] The feature extraction module 810 is used to acquire the image to be located, perform image feature extraction on the image to be located, and acquire the positioning image features corresponding to the image to be located. The positioning image feature vector includes low-dimensional image features and high-dimensional image features.

[0196] The preliminary matching module 820 is used to match low-dimensional image features with multiple index features in the image location library to obtain at least one candidate index feature that matches the low-dimensional image features; the image location library stores the location image features corresponding to multiple location images, the index features constructed based on the location image features, and the geographical location information associated with the location images;

[0197] The fine matching module 830 is used to match the high-dimensional image features with the location image features corresponding to the candidate index features in response to the candidate index features not meeting the preset conditions, so as to determine the real geographical location information of the image to be located.

[0198] In the positioning device provided in this embodiment, feature extraction from images is combined with feature matching technology to achieve accurate matching between the image to be located and the image features in the image location database, thereby achieving accurate positioning. Since it does not rely on external devices (such as GPS) and can locate solely through image features, accurate positioning can still be achieved in indoor and underground scenes where GPS is unavailable. Because image features are more resistant to interference from lighting and seasonal changes, they are more robust and adaptable to different scenes. Simultaneously, by using low-dimensional image features to quickly narrow down the matching range and high-dimensional image features to ensure accurate hierarchical matching, it avoids both the "waste of resources from full high-dimensional computation" and the "insufficient accuracy of single low-dimensional matching."

[0199] In some possible implementations, the positioning device further includes a matching processing module, which, in response to the candidate index features satisfying preset conditions, determines at least one candidate geographic location information corresponding to the image to be positioned based on the geographic location information associated with the location image corresponding to the candidate index features; and determines the actual geographic location information of the image to be positioned based on the at least one candidate geographic location information.

[0200] In some possible implementations, the fine matching module is specifically used to: match the high-dimensional image features with the location image features corresponding to the candidate index features to obtain at least one location image feature that matches the high-dimensional image features; determine at least one candidate geographic location information corresponding to the image to be located based on the geographic location information associated with the location image corresponding to the location image feature; and determine the true geographic location information of the image to be located based on at least one candidate geographic location information.

[0201] In some possible implementations, the preliminary matching module and the fine matching module include a semantic matching unit for: performing semantic verification on at least one candidate geographic location information, and determining the true geographic location information of the image to be located based on the semantic verification result.

[0202] In some possible implementations, semantic verification includes multimodal semantic verification; the semantic matching unit is specifically used to: extract text information from the image to be located, calculate the semantic similarity between the text information and the candidate geographic location information, and filter the candidate geographic location information based on the semantic similarity.

[0203] In some possible implementations, semantic verification includes scene semantic verification; the semantic matching unit is specifically used to: identify the image scene corresponding to the image to be located; and filter the candidate geographic location information based on the consistency between the image scene and the image scene corresponding to the candidate geographic location information.

[0204] In some possible implementations, the positioning device also includes a database construction module for: acquiring multiple location images and the geographic location information associated with the location images; extracting image features from the location images to obtain the location image features corresponding to the location images, and constructing index features based on the location image features; and storing the geographic location information associated with the location images, the location image features corresponding to the location images, and the index features corresponding to the location images together to construct an image location database.

[0205] In some possible implementations, the positioning device further includes an index structure construction module for: constructing a first index structure based on the index features corresponding to the location image; constructing a second index structure based on the location image features corresponding to the location image; a preliminary matching module for: searching for candidate index features similar to low-dimensional image features in the first index structure; and a fine matching module for: searching for location image features corresponding to candidate index features based on high-dimensional image features in the second index structure.

[0206] In some possible implementations, the index structure building module is used to: cluster the index features corresponding to all location images using a clustering algorithm to form multiple cluster centers; and assign each index feature to its respective cluster center to form the first index structure.

[0207] In some possible implementations, the second index structure is a graph-based nearest neighbor index structure.

[0208] In some possible implementations, the positioning device further includes a data block update module for: monitoring and acquiring newly added location images and their associated geographic location information; extracting image features from the newly added location images to obtain the location image features corresponding to the newly added location images, and constructing index features based on the location image features; associating and adding the geographic location information associated with the newly added location images, the location image features corresponding to the newly added location images, and the corresponding index features to the image library; incrementally updating the first index structure based on the index features corresponding to the newly added location images; and incrementally updating the second index structure based on the location image features corresponding to the newly added location images.

[0209] In some possible implementations, the positioning device also includes a hotspot data module for: statistically analyzing frequently accessed geographic locations based on historical data; constructing a hotspot data index based on the location image features corresponding to the location images of the frequently accessed geographic locations; and matching the location image features with the location image features in the hotspot data index.

[0210] In some possible implementations, the feature extraction module is used to: extract image features from the location image using a deep convolutional neural network optimized with a metric learning loss function, and obtain the location image features corresponding to the location image; extract image features from the location image using a visual transformer model, and obtain the location image features corresponding to the location image; extract image features from the location image using a visual model pre-trained based on self-supervised learning, and obtain the location image features corresponding to the location image; and extract image features from the location image using a multimodal contrastive learning model, and obtain the location image features corresponding to the location image.

[0211] In some possible implementations, the feature extraction module is used to: normalize the location image features corresponding to the acquired location image; quantize the normalized location image features to construct index features.

[0212] It is understood that the above-described modules of the positioning device in the embodiments of this disclosure have the ability to implement... Figure 1 The positioning method in the illustrated embodiment demonstrates the functionality of corresponding steps. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. These modules can be software and / or hardware, and each module can be implemented individually or integrated from multiple modules. For a detailed description of the functions of each module in the aforementioned positioning device, please refer to [link to relevant documentation]. Figure 1 The corresponding descriptions of the positioning methods in the embodiments shown are not repeated here.

[0213] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0214] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0215] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0216] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the positioning method provided in the embodiments of this disclosure.

[0217] Compared to existing technologies, this electronic device achieves precise positioning by extracting features from images and combining them with feature matching technology to accurately match the image to be located with image features in an image location database. Since it does not rely on external devices (such as GPS) and can locate solely through image features, it can still achieve accurate positioning in indoor and underground environments where GPS is unavailable. Because image features are more resistant to interference from lighting and seasonal changes, it is more robust and adaptable to different scenarios. Furthermore, by using low-dimensional image features to quickly narrow down the matching range and high-dimensional image features to ensure accurate hierarchical matching, it avoids both the waste of resources from full high-dimensional computation and the insufficient accuracy of single low-dimensional matching.

[0218] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the positioning method provided in the embodiments of this disclosure.

[0219] Compared to existing technologies, this readable storage medium achieves precise positioning by extracting features from images and combining them with feature matching techniques to accurately match the image to be located with image features in an image location database. Since it does not rely on external devices (such as GPS) and can locate solely through image features, it can still achieve accurate positioning in indoor or underground environments where GPS is unavailable. Because image features are more resistant to interference from lighting and seasonal changes, it is more robust and adaptable to different scenarios. Furthermore, by using low-dimensional image features to quickly narrow down the matching range and high-dimensional image features to ensure accurate hierarchical matching, it avoids both the waste of resources from full high-dimensional computation and the insufficient accuracy of single low-dimensional matching.

[0220] The computer program product includes a computer program that, when executed by a processor, implements the positioning method as provided in the embodiments of this disclosure.

[0221] Compared to existing technologies, this computer program product achieves precise positioning by extracting features from images and combining them with feature matching technology to accurately match the image to be located with image features in an image location database. Because it does not rely on external devices (such as GPS) and can locate solely through image features, it can still achieve accurate positioning in indoor and underground scenes where GPS is unavailable. Furthermore, image features are more resistant to interference from lighting and seasonal changes, resulting in greater robustness and scene adaptability. Simultaneously, by using low-dimensional image features to quickly narrow down the matching range and high-dimensional image features to ensure accurate hierarchical matching, it avoids both the "waste of resources from full high-dimensional computation" and the "insufficient accuracy of single low-dimensional matching."

[0222] Figure 9 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0223] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0224] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0225] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the positioning method. For example, in some embodiments, the positioning method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the positioning method by any other suitable means (e.g., by means of firmware).

[0226] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0227] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0228] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0229] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0230] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0231] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0232] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0233] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A positioning method, comprising: obtaining a to-be-positioned image, performing image feature extraction on the to-be-positioned image, and obtaining a positioning image feature corresponding to the to-be-positioned image, the positioning image feature vector comprising a low-dimensional image feature and a high-dimensional image feature; performing matching on the low-dimensional image feature and a plurality of index features of an image location library, and obtaining at least one candidate index feature matched with the low-dimensional image feature; the image location library storing a plurality of location image features corresponding to location images, index features constructed according to the location image features, and geographic location information associated with the location images; in response to the candidate index feature not satisfying a preset condition, performing matching on the high-dimensional image feature and a location image feature corresponding to the candidate index feature, and determining real geographic location information of the to-be-positioned image.

2. The method of claim 1, further comprising: in response to the candidate index feature satisfying a preset condition, determining at least one candidate geographic location information corresponding to the to-be-positioned image according to geographic location information associated with a location image corresponding to the candidate index feature; determining real geographic location information of the to-be-positioned image according to at least one candidate geographic location information.

3. The method of claim 1, wherein, the step of determining real geographic location information of the to-be-positioned image by performing matching on the high-dimensional image feature and a location image feature corresponding to the candidate index feature, comprising: performing matching on the high-dimensional image feature and at least one location image feature corresponding to the candidate index feature, and obtaining at least one candidate geographic location information corresponding to the to-be-positioned image according to geographic location information associated with a location image corresponding to the location image feature; determining real geographic location information of the to-be-positioned image according to at least one candidate geographic location information. the step of determining real geographic location information of the to-be-positioned image according to at least one candidate geographic location information, comprising:

4. The method of claim 2 or 3, wherein, performing semantic verification on at least one candidate geographic location information, and determining real geographic location information of the to-be-positioned image based on a semantic verification result. the semantic verification comprises multi-modal semantic verification; the step of performing semantic verification on at least one candidate geographic location information, and determining real geographic location information of the to-be-positioned image based on a semantic verification result, comprising:

5. The method of claim 4, wherein, extracting text information in the to-be-positioned image, and calculating semantic similarity of the text information and the candidate geographic location information; screening the candidate geographic location information according to the semantic similarity. the semantic verification comprises scene semantic verification; the step of performing semantic verification on at least one candidate geographic location information, and determining real geographic location information of the to-be-positioned image based on a semantic verification result, comprising:

6. The method of claim 4, wherein, identifying an image scene corresponding to the to-be-positioned image; screening the candidate geographic location information according to consistency of the image scene and an image scene corresponding to the candidate geographic location information.

7. The method of claim 1, further comprising: obtaining a plurality of location images and geographic location information associated with the location images. ​ perform image feature extraction on the location images to obtain location image features corresponding to the location images, and construct index features according to the location image features; store the geographic location information associated with the location images, the location image features corresponding to the location images, and the index features corresponding to the location images in association, and construct an image location library.

8. The method of claim 7, further comprising: constructing a first index structure according to the index features corresponding to the location images; constructing a second index structure according to the location image features corresponding to the location images; the matching of the low-dimensional image features with the plurality of index features of the image location library comprises searching for candidate index features similar to the low-dimensional image features in the first index structure; the matching of the high-dimensional image features with the location image features corresponding to the candidate index features comprises searching for the location image features corresponding to the candidate index features based on the high-dimensional image features in the second index structure.

9. The method of claim 8, wherein, the construction of the first index structure according to the index features corresponding to the location images comprises: performing clustering on all the index features corresponding to the location images using a clustering algorithm to form a plurality of cluster centers; assigning each of the index features to a cluster center to which the index feature belongs to form the first index structure.

10. The method of claim 8, wherein, the second index structure is a graph-based neighbor index structure.

11. The method of claim 8, further comprising: monitoring and obtaining newly added location images and geographic location information associated with the location images; performing image feature extraction on the newly added location images to obtain location image features corresponding to the newly added location images, and constructing index features according to the location image features; associating and adding the geographic location information associated with the newly added location images, the location image features corresponding to the newly added location images, and the corresponding index features to the image library; incrementally updating the first index structure based on the index features corresponding to the newly added location images; incrementally updating the second index structure based on the location image features corresponding to the newly added location images.

12. The method of claim 7, further comprising: statistically determining high-frequency access geographic locations according to historical data; constructing a hotspot data index according to location image features corresponding to the high-frequency access geographic locations; after obtaining the positioning image features corresponding to the to-be-positioned image, the method further comprises: matching the positioning image features with the location image features in the hotspot data index.

13. The method of claim 7, wherein, the image feature extraction on the location images to obtain location image features corresponding to the location images comprises at least one of the following: performing image feature extraction on the location images using a deep convolutional neural network optimized by a metric learning loss function to obtain location image features corresponding to the location images; performing image feature extraction on the location images using a visual transformer model to obtain location image features corresponding to the location images; perform image feature extraction on the position image using a visual model pre-trained based on self-supervised learning to obtain position image features corresponding to the position image; perform image feature extraction on the position image using a multi-modal contrast learning model to obtain position image features corresponding to the position image.

14. The method of claim 13, wherein, The image feature extraction on the position image to obtain position image features corresponding to the position image and the construction of index features according to the position image features include: perform normalization processing on the obtained position image features corresponding to the position image; perform quantization processing on the normalized position image features to construct the index features.

15. A positioning apparatus, comprising: a feature extraction module configured to obtain a to-be-positioned image, perform image feature extraction on the to-be-positioned image, and obtain positioning image features corresponding to the to-be-positioned image, the positioning image feature vector including low-dimensional image features and high-dimensional image features; a preliminary matching module configured to match the low-dimensional image features with a plurality of index features of an image position library to obtain at least one candidate index feature matched with the low-dimensional image features; the image position library stores position image features corresponding to a plurality of position images, index features constructed according to the position image features, and geographical position information associated with the position images; a fine matching module configured to, in response to the candidate index feature not satisfying a preset condition, match the high-dimensional image features with position image features corresponding to the candidate index feature to determine real geographical position information of the to-be-positioned image.

16. An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-14.

17. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-14.

18. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-14. The computer program product comprises a computer program which, when executed by a processor, implements the method of any one of claims 1-14.

Images