Geospatial position location method, apparatus, device, and product

Abstract

This application provides a geospatial location positioning method, apparatus, device, and product, relating to the field of map positioning technology. The method includes: receiving multimodal data, including initial coordinates, image data, and text data; inputting the multimodal data into a multimodal large language model to obtain visual fingerprint features; determining the corresponding target road based on the initial coordinates; acquiring candidate images corresponding to multiple candidate points on the target road, and performing semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to select the target point from the multiple candidate points; and determining the target location based on the target point. By fusing multimodal data and utilizing a multimodal large language model to extract visual fingerprint features, combined with road constraints and semantic consistency matching, highly adaptable, efficient, and accurate geospatial location positioning is achieved in complex real-world environments.

Description

Technical Field

[0001] This application relates to the field of map positioning technology, and in particular to a geospatial location positioning method, device, equipment and product. Background Technology

[0002] With the rapid development of smart cities, autonomous driving and geographic information services, higher requirements are being placed on the accuracy, efficiency and environmental adaptability of geospatial positioning.

[0003] In related technologies, geospatial positioning often relies on path search with fixed step sizes or region traversal methods based on preset rules. For example, in path planning or target search tasks, candidate points are often generated by sampling at equal intervals or expanding with a fixed radius, and then the location is determined by filtering according to rules.

[0004] However, the above methods and procedures are relatively rigid and difficult to cope with complex and ever-changing real-world environments, resulting in poor adaptability, low efficiency, and poor positioning accuracy. Summary of the Invention

[0005] This application provides a geospatial location positioning method, device, equipment, and product. By fusing multimodal data and extracting visual fingerprint features using a multimodal large language model, and combining road constraints and semantic consistency matching, it achieves highly adaptable, efficient, and accurate geospatial location positioning in complex real-world environments.

[0006] Firstly, this application provides a geospatial location positioning method, the method comprising:

[0007] Receive multimodal data, which includes initial coordinates, image data, and text data;

[0008] Multimodal data is input into a multimodal large language model to obtain visual fingerprint features;

[0009] Determine the corresponding target road based on the initial coordinates;

[0010] Acquire candidate images corresponding to multiple candidate points on the target road, and perform semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to select the target point from the multiple candidate points;

[0011] Determine the target location based on the target point.

[0012] Secondly, this application provides a geospatial location positioning device, the device comprising:

[0013] The receiving module is used to receive multimodal data, which includes initial coordinates, image data, and text data.

[0014] The input module is used to input multimodal data into a multimodal large language model to obtain visual fingerprint features;

[0015] The first determining module is used to determine the corresponding target road based on the initial coordinates;

[0016] The matching module is used to obtain candidate images corresponding to multiple candidate points on the target road, and to perform semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to select the target point from multiple candidate points.

[0017] The second determination module is used to determine the target location based on the target point.

[0018] Thirdly, this application provides an electronic device, including: a memory and a processor;

[0019] The memory stores the instructions that the computer executes;

[0020] The processor executes computer execution instructions stored in memory, causing the processor to perform the method as described in any of the first aspects.

[0021] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the first aspects.

[0022] The geospatial location positioning method, apparatus, equipment, and product provided in this application receive multimodal data, including initial coordinates, image data, and text data. Furthermore, they utilize a multimodal large language model to parse the multimodal data. By introducing a multimodal large language model to process multi-source heterogeneous data such as initial coordinates, image data, and text data, they extract visual fingerprint features with semantic information, replacing traditional fixed step size or preset rule methods. This enables the positioning process to adapt to complex and ever-changing real-world environments and improves adaptability. Furthermore, the target road is determined based on the initial coordinates, such as whether it is a regular road segment or an intersection. Then, candidate images of multiple candidate points on the target road are obtained, and their semantic features are extracted. Further, the semantic features of the candidate images corresponding to multiple candidate points on the target road are matched with visual fingerprint features. The target point is selected through semantic consistency, and the target location is determined based on the target point. This method of quickly determining the target road based on the initial coordinates only requires semantic matching of the candidate points on the road, avoiding global traversal or blind search, reducing the calculation range, improving positioning efficiency, and filtering out target points that match the visual fingerprint features through the fusion of multimodal data and semantic consistency matching, achieving higher accuracy in location determination, reducing positioning deviations caused by environmental changes or fixed rules, and further improving positioning accuracy. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0025] Figure 2 A flowchart illustrating a geospatial location positioning method provided in an embodiment of this application;

[0026] Figure 3 A flowchart illustrating an optional geospatial location positioning method provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the structure of a positioning system provided in an embodiment of this application;

[0028] Figure 5 An interactive schematic diagram illustrating a geospatial location positioning method provided in an embodiment of this application;

[0029] Figure 6 A schematic diagram of a geospatial location positioning device provided in an embodiment of this application;

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0031] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] First, let me explain the terms used in this application:

[0035] Intelligent agent: can refer to an autonomous entity with perception, decision-making and execution capabilities, such as autonomous vehicles, drones or robots;

[0036] Multimodal Large Language Model (VLM): refers to an artificial intelligence model that can simultaneously understand, process, and generate multiple modalities of information such as text, images, audio, and video.

[0037] Visual fingerprint: refers to the set of core semantic information extracted by VLM after unified parsing and structuring of multimodal data. It can include semantic information such as target features, road layout, and distribution of reference objects, and can be used as benchmark data for subsequent comparison and matching.

[0038] Topology-aware refers to the ability of a positioning system to identify the topology of its current environment (e.g., whether it is an intersection or a regular road segment) and to dynamically adjust its search path and evaluation strategy based on the identification results.

[0039] Road network topology: refers to the logical abstract set of connections and spatial layouts between various road elements (such as road segments and intersections) in a road network. It is used to describe the interconnection and intersection between different parts of the road network and the overall form of the network.

[0040] In one possible implementation, geospatial positioning often relies on path search with a fixed step size or region traversal methods based on preset rules. For example, in path planning or target search tasks, candidate points are often generated by sampling at equal intervals or expanding with a fixed radius, and then the location is determined by filtering according to rules.

[0041] Although the above methods are simple to implement, have low hardware requirements, and are highly adaptable, they typically use a fixed exploration step size and cannot be adaptively adjusted according to the actual terrain or scene complexity (such as open areas or narrow alleys). The methods are relatively rigid and difficult to cope with complex and ever-changing real-world environments, resulting in poor adaptability, low efficiency, and poor positioning accuracy, especially in densely populated urban areas where the misjudgment rate is high.

[0042] For example, in open areas, a fixed step size can lead to redundant sampling points, wasting computational or sensing resources; in narrow or complex areas, an excessively large step size may cause key details or features to be missed. For these reasons, its overall exploration efficiency fluctuates greatly, and its performance is unpredictable.

[0043] Another possible implementation could be based on The algorithm performs path planning, which involves calculating the optimal path using preset weight parameters (such as path length and obstacle density), and then using the calculated optimal path and the current position to locate the target and determine the final location result.

[0044] While this algorithm is highly efficient, responds quickly, and generates smooth, continuous optimal paths, it relies entirely on preset, fixed weight parameters. It cannot perceive changes in the actual environment (such as traffic flow, obstacles, and intersection complexity), thus failing to dynamically adjust search strategies or exploration intensity and lacking environmental awareness and adaptive capabilities. Especially in complex intersections and similar environments, these limitations easily lead to misjudgments. This typically manifests as a high average deviation rate, meaning a large average error between the optimal path or location result and the actual situation, resulting in low accuracy of the location results.

[0045] To address the aforementioned issues, this application provides a geospatial location positioning method. By receiving multimodal data, including initial coordinates, image data, and text data, and further utilizing a multimodal large language model to parse the multimodal data, this method processes multi-source heterogeneous data such as initial coordinates, image data, and text data using a multimodal large language model. It extracts visual fingerprint features with semantic information, replacing traditional methods with fixed step sizes or preset rules. This enables the positioning process to adapt to complex and ever-changing real-world environments, improving its adaptability. Furthermore, the target road is determined based on the initial coordinates, such as whether it is a regular road segment or an intersection. Then, candidate images of multiple candidate points on the target road are obtained, and their semantic features are extracted. Further, the semantic features of the candidate images corresponding to multiple candidate points on the target road are matched with visual fingerprint features. The target point is selected through semantic consistency, and the target location is determined based on the target point. This method of quickly determining the target road based on the initial coordinates only requires semantic matching of the candidate points on the road, avoiding global traversal or blind search, reducing the calculation range, improving positioning efficiency, and filtering out target points that match the visual fingerprint features through the fusion of multimodal data and semantic consistency matching, achieving higher accuracy in location determination, reducing positioning deviations caused by environmental changes or fixed rules, and further improving positioning accuracy.

[0046] For example, Figure 1 This is a schematic diagram of an application scenario provided in an embodiment of this application, such as... Figure 1 As shown, this application takes the application of geospatial positioning to autonomous vehicles as an example. The application scenario includes a vehicle 101 and a positioning system 102. The vehicle 101 is equipped with map software. The positioning system 102 can be integrated into the control system of the vehicle 101, or it can be integrated into the cloud or a remote server. This application embodiment does not specifically limit this.

[0047] The positioning system 102 acquires multimodal data collected in real time by the vehicle 101, including initial coordinates provided by the vehicle 101's own positioning system, image data captured by the onboard camera, and potentially associated text data. This text data can be navigation instructions, road name information, or environmental text parsed by vehicle sensors. It can also be text data input by the user through map software on the vehicle 101, or text data directly input by the user to the vehicle's infotainment system. This application embodiment does not specifically limit the specific content and source of the text data; the above is merely illustrative.

[0048] Furthermore, the positioning system 102 inputs the aforementioned multimodal data into a multimodal large language model, which fuses image content and text semantics to obtain visual fingerprint features that comprehensively represent the current scene. Based on the initial coordinates provided by the vehicle 101, it matches and determines the target road where the vehicle 101 is currently located in the map data.

[0049] Furthermore, acquire candidate images (such as street view images and / or panoramic map images) corresponding to multiple candidate points pre-collected or generated in real time on the target road, extract the semantic features of each candidate image, perform semantic consistency matching with the visual fingerprint features determined by vehicle 101, and select the semantically matching candidate points as target points.

[0050] In this way, the coordinates of the selected target points can be used as the corrected and accurate positioning result of vehicle 101, thereby achieving high-precision spatial positioning of vehicle 101 in complex environments.

[0051] It should be noted that the embodiments of this application do not specifically limit the application scenarios of this application. The above are just examples. For example, this application scenario can also be applied to the last mile of navigation, where accurate positioning can be achieved through received image data and text data; it can also be applied to user feedback problem scenarios, where the problem location can be accurately located based on the image data and text data uploaded by the user; or it can be an indoor positioning scenario, etc.

[0052] For example, taking the last mile of navigation as an example, in an autonomous driving scenario, when vehicle 101 enters the last mile range of the destination (e.g., a radius of 500 meters), the positioning system 102 acquires the initial coordinates of vehicle 101, real-time image data captured by onboard sensors, and the text data "XX Building B2 Parking Lot Entrance" input by the user. The initial coordinates, image data, and text data are input into a multimodal large language model, and a unified visual fingerprint feature is generated through model fusion processing to comprehensively represent the semantic and spatial information of the current scene.

[0053] If the road segment located near the parking lot entrance is determined based on the initial coordinates, then this road segment is identified as the target road. Multiple candidate points are then selected on the target road, and their pre-stored candidate images (such as street view images or panoramic map images) are obtained. Semantic features of the candidate images are extracted and semantically matched with visual fingerprint features to filter out the target point from the multiple candidate points.

[0054] The geographic coordinates corresponding to the target point are used as the final positioning result, i.e., the precise location of "Entrance to Parking Lot B2 of XX Building". Optionally, the target location can also be synchronized to the vehicle map to update the current position coordinates of vehicle 101 in the map coordinate system, completing the precise positioning of the last mile of navigation.

[0055] Alternatively, taking a user feedback scenario as an example, the user's photos, input text, and initial coordinates of the points marked on the map software are reported to the map software's backend system. The text indicating that traffic signs or other information at the marked location are inaccurate or have been updated can then be used by the backend system to determine the target location and send that location back to the map software for updating.

[0056] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0057] For example, Figure 2 This is a flowchart illustrating a geospatial location positioning method provided in an embodiment of this application, as shown below. Figure 2 As shown, the executing entity of this geospatial location positioning method can be the aforementioned positioning system, an intelligent agent, or the cloud. This application embodiment does not specifically limit this. The geospatial location positioning method includes the following steps:

[0058] S201. Receive multimodal data, which includes initial coordinates, image data, and text data.

[0059] In this embodiment, multimodal data can refer to a collection of raw data from different perceptual modalities, including initial coordinates, image data, and text data.

[0060] The initial coordinates can refer to rough geographic location information, either manually marked by the user or automatically generated by the device. This information can come from the Global Positioning System (GPS) or cellular network positioning, and is used to provide an initial reference range for spatial search or inference for the positioning system, rather than a precise spatial positioning result, which may contain biases. This application does not limit the specific content corresponding to the initial coordinates.

[0061] Image data can refer to environmental images containing scene visual information collected by users or devices, such as street view images, road sign images, and building images, used to provide environmental visual features. This application does not limit the specific content corresponding to the image data.

[0062] Text data can refer to text descriptions input by the user or associated with the device, such as location names, road sign text, environmental notes, etc., used to supplement semantic information. This application does not limit the specific content corresponding to the text data.

[0063] In this step, the positioning system can simultaneously receive multimodal data from different sensors or sources, including initial coordinates, image data, and text data, as the raw information input for subsequent positioning processing.

[0064] It should be noted that the above-mentioned device may be an intelligent agent, a terminal device, etc. This application embodiment does not specifically limit it. In the following embodiments, a vehicle is usually used for explanation and description, but it is not limited to a vehicle.

[0065] S202. Input the multimodal data into the multimodal large language model to obtain visual fingerprint features.

[0066] In this embodiment, the visual fingerprint feature refers to a data representation generated by a multimodal large language model that provides a structured and semantic description of the current visual environment. It may include semantic information features crucial for localization, such as target features, road layout, and the distribution of reference points. It should be noted that this visual fingerprint feature can be used to characterize the semantic features of multimodal data.

[0067] In this step, the localization system inputs the received multimodal data (initial coordinates, image data, and text data) into the multimodal large language model. The multimodal large language model performs joint understanding and fusion of image content and text semantics, and outputs a feature representation that can comprehensively represent the semantic information of the current scene, namely, the visual fingerprint feature.

[0068] For example, the positioning system inputs the received multimodal data—initial coordinates (coordinates of a point in an intersection area), image data (images containing the scene of the intersection area), and text data (such as a user description "at the northeast corner of the intersection, there is a red building next to it, and a bus stop next to the red building, but the bus stop is marked at the southeast corner of the intersection on the map")—into a multimodal large language model to obtain visual fingerprint features. These visual fingerprint features semantically represent "a location located at the northeast corner of an intersection, whose visual scene contains a red building as its core element, and a bus stop next to the red building."

[0069] S203. Determine the corresponding target road based on the initial coordinates.

[0070] In this embodiment, the target road refers to a road unit in the environment where the device is located, inferred from the initial coordinates. It is used to determine the topological connectivity of the road network and can be a specific road segment or intersection. This target road serves as the spatial constraint range for subsequent candidate point search and matching.

[0071] In this step, the positioning system can determine the road unit associated with the initial coordinates, i.e. the target road, in the geographic information system based on the initial coordinates through map matching or spatial indexing technology, thereby converging the positioning search range from the global area to the specific road.

[0072] S204. Obtain candidate images corresponding to multiple candidate points on the target road, and perform semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to select the target point from the multiple candidate points.

[0073] In this embodiment of the application, candidate images may refer to image data collected or generated at various candidate point locations on the target road, such as street view images and / or panoramic map images, used to represent the visual scene of the candidate points.

[0074] Street view images can refer to images that simulate a real field of view, acquired or rendered from the candidate point location and along the road direction. Panoramic map images can refer to top-down or panoramic views generated from the candidate point location, containing complete 360-degree information about the surrounding environment. They provide global spatial context and topological relationships, helping to verify whether the candidate point matches the overall layout of surrounding roads and buildings, compensating for the limited field of view of street view images.

[0075] Semantic features can refer to vectorized features extracted from candidate images that can represent high-level semantic information of their visual content.

[0076] Semantic consistency matching refers to the process of evaluating the degree of semantic consistency between semantic features and visual fingerprint features by calculating the similarity in the semantic space, and then selecting the optimal matching point based on this degree of consistency. For example, semantic consistency matching can be implemented using a Large Language Model (LLM), which inputs the semantic features of candidate images corresponding to multiple candidate points and the visual fingerprint features obtained from S202 into the LLM to obtain the semantically best matching candidate point, which is then used as the target point.

[0077] Optionally, the method for semantic consistency matching can also be a method based on a neural network encoder, a method for calculating the Jaccard similarity coefficient, etc. The embodiments of this application do not specifically limit the method for performing semantic consistency matching.

[0078] In this step, the positioning system can acquire candidate images corresponding to multiple candidate points pre-collected or generated on the target road, then extract the semantic features of each candidate image, perform similarity calculation and matching with visual fingerprint features, and select the most matching target point from the candidate points based on the degree of semantic consistency.

[0079] For example, suppose the initial coordinates are located at an intersection, and the visual fingerprint features indicate "there is a red building at the northeast corner of the intersection, and a bus stop is located next to the red building." The semantic consistency matching process includes: taking an intersection with a connecting road as an example, for multiple candidate points on the connecting road in the intersection area, extract the semantic features of their candidate images, such as "northeast corner," "red building," "bus stop," and "intersection." Further, by calculating the similarity between the semantic features corresponding to each candidate image and the aforementioned visual fingerprint features, the candidate point with the highest semantic consistency can be selected. For example, if both semantic features contain "red building" and "bus stop," and the image is located in the northeast direction of the intersection, then this candidate point is determined as the target point.

[0080] It should be noted that if there are multiple connecting roads in the intersection area, similar execution steps as described above can be performed to determine the candidate point with the highest semantic consistency. The specific process will not be elaborated in this embodiment.

[0081] S205. Determine the target location based on the target point.

[0082] In this embodiment, the target location may refer to the precise geographic coordinates corresponding to the target point selected through semantic consistency matching. It is a spatial positioning result that represents the spatial location of the real scene described by the multimodal data. Optionally, the spatial positioning result may include geographic coordinates (such as latitude and longitude, altitude) and attitude information (such as orientation). This embodiment does not limit the specific content corresponding to the spatial positioning result.

[0083] For example, the positioning system uses the geographic coordinates corresponding to the selected target points as the final positioning result, thus completing the correction and determination from the initial coordinates to the target location.

[0084] Thus, this application extracts visual fingerprint features with fused semantics through a multimodal large language model, enabling the localization process to operate independently of fixed rules and instead match based on semantic understanding of the actual scene. This allows it to adapt to different terrains and scene complexities (such as open areas and narrow alleys) without the need for pre-set adjustment strategies, enhancing the method's adaptability to complex and changing environments. Furthermore, based on initial coordinate constraints on the target road, target points are filtered through semantic consistency matching, avoiding blind global search. Semantic consistency matching also filters target points that match the multimodal features from road candidate points, achieving fine-grained localization based on semantic understanding. This reduces errors caused by environmental interference or rigid rules, improving localization accuracy.

[0085] Furthermore, this application utilizes initial coordinates to quickly converge the search range to the target road, requiring only semantic matching of a limited number of candidate points. This avoids inefficient region traversal or path search, significantly reducing computational load and improving positioning efficiency.

[0086] Optionally, the processing procedure of the multimodal large language model for multimodal data includes:

[0087] By using a multimodal large language model, road topology features corresponding to the initial coordinates, target features in image data, and semantic description features in text data are extracted. The road topology features, semantic description features, and target features are then fused to obtain visual fingerprint features.

[0088] In this embodiment of the application, road topology features can refer to structured features extracted from the road network associated with the initial coordinates, which characterize the spatial topology information such as the connection relationship, direction and shape of the roads.

[0089] Target features can refer to visual features extracted from image data, which characterize key objects, scene composition, and visual attributes in the image. For example, they can characterize the appearance, category, and location information of specific visual instances in the image (such as traffic signs, building outlines, and lane lines).

[0090] Semantic descriptive features refer to semantic features extracted from text data, which represent keywords, context, and semantic information in the text description.

[0091] In this step, the multimodal large language model extracts road topology features from the initial coordinates, target features from the image data, and semantic description features from the text data. Then, these heterogeneous features are fused to obtain a unified visual fingerprint feature.

[0092] The fusion process involves aligning and integrating road topology features, target features, and semantic description features across modalities using a multimodal large language model. This application does not specify the method used for the fusion process.

[0093] In this way, by extracting and fusing road topology features, semantic description features, and target features separately, the resulting visual fingerprint features provide a more comprehensive description of the environment, are more resistant to noise from a single modality, and enhance the richness and robustness of feature representation. Furthermore, by extracting representative high-level features from each modality separately and then fusing them, more accurate and deeper semantic alignment and information complementarity can be achieved compared to directly processing the raw data, realizing deep cross-modal fusion. Therefore, by obtaining structured multimodal visual fingerprint features, unstructured input is transformed into quantifiable representations, enabling complementary utilization of location, visual, and textual information, enhancing the representational power of features, thereby improving cross-modal consistency and enhancing matching reliability.

[0094] Optionally, the target road can be determined based on the initial coordinates, including:

[0095] In response to the initial coordinates being located in the intersection area, multiple connecting roads in the intersection area are identified as target roads;

[0096] If the initial coordinates are located in a road segment area, then the road segment containing that area is determined as the target road.

[0097] In this embodiment, the intersection area can refer to the intersection of two or more roads and its adjacent buffer zone. It is characterized by complex topological structure, variable visual features, and ambiguity in travel direction.

[0098] Connecting roads can refer to road segments in various directions that are directly connected to a certain intersection.

[0099] A road segment area can refer to a single road segment between intersections that has continuous traffic characteristics and the area along its route. Its characteristics include a relatively simple environmental topology and clear directions.

[0100] In this step, the scope of the target road can be dynamically determined based on the geographical location type (intersection or road segment) of the initial coordinates: if the initial coordinates are located at an intersection, all roads connected to that intersection are included in the target road set; if the initial coordinates are located at a road segment, only that road segment itself is considered as the target road.

[0101] In this way, this application can adaptively adjust the road search range according to the coordinate location type, avoiding omissions or misjudgments caused by the multiple directions of intersections. Furthermore, while ensuring coverage of possible locations, it minimizes the candidate road set as much as possible, improving subsequent matching efficiency.

[0102] Optionally, in response to the initial coordinates being located in the intersection area, semantic consistency matching is performed between the semantic features of the candidate images and the visual fingerprint features to filter out the target point from multiple candidate points, including:

[0103] For each connecting road, multiple candidate points for the connecting road are determined;

[0104] Using the semantic features and visual fingerprint features of candidate images of multiple candidate points, semantic consistency is evaluated and ranked for each candidate point to determine the first and second candidate points.

[0105] The candidate images of the first and second candidate points are packaged to generate composite candidates, and the composite candidates corresponding to the target road are filtered to obtain the target point.

[0106] In this embodiment, the target road corresponding to the intersection area can refer to the set of all roads directly connected to the intersection and accessible by car, listed according to map topology data. It defines all possible directional assumptions when performing location search at the intersection and serves as the basis for multi-path candidate point sampling.

[0107] Optionally, candidate points are sampled for each connecting road to obtain multiple candidate points. Candidate points can refer to potential matching location points that represent the direction of each connecting road in the target road, generated by sampling according to certain rules (such as distance intervals).

[0108] Optionally, the first candidate point (best point) can refer to the candidate point with the highest semantic consistency evaluation score with visual fingerprint features on a single connected path. The second candidate point (second best point) can refer to the candidate point with the second highest semantic consistency evaluation score with visual fingerprint features on a single connected path.

[0109] It should be noted that selecting the first and second candidate points as the best and second best points is to address potential output fluctuations in the semantic matching process of the multimodal large language model. This is because the model output may differ each time. For example, for the same input data, the first output might show candidate point 1 as the best point and candidate point 2 as the second best point, while the second output might show candidate point 2 as the best point and candidate point 1 as the second best point. Therefore, selecting the best and second best points ensures that competitive alternatives are retained in the candidate point selection for a single road, thus improving fault tolerance.

[0110] Composite candidates refer to a set of candidate images packaged from the first and second candidate points of each connecting road at the same intersection, used to represent multi-directional candidate information for the intersection area. Its design aims to ensure output quality while preserving result diversity, and to support subsequent candidate comparison and selection in a "one-on-one elimination" format.

[0111] For example, when the initial coordinates are located in an intersection area, the positioning system can extract multiple candidate points for each connecting road, and select the first and second candidate points for each road through semantic consistency evaluation and ranking. Further, the candidate images of the first and second candidate points are packaged to generate composite candidates, and then a comprehensive screening is performed based on the composite candidates to determine the target point. Here, semantic consistency evaluation and ranking can refer to the process of scoring candidate points and ranking them according to their scores by calculating the similarity between the semantic features and visual fingerprint features of the candidate images.

[0112] Optionally, semantic consistency evaluation and ranking are performed on each candidate point using the semantic features and visual fingerprint features of the candidate images of multiple candidate points to determine the first candidate point and the second candidate point, including:

[0113] The model scoring criteria for LLM are constructed based on visual fingerprint features. The semantic features and visual fingerprint features of candidate images of multiple candidate points are input into LLM for processing, so as to calculate the matching score of each candidate point in each road based on the model scoring criteria. Based on the matching score, each candidate point of each road is sorted, and the candidate point with the highest and second highest scores in each road is selected as the first candidate point and the second candidate point of the road.

[0114] The model scoring criteria can refer to a set of dynamic standard rules or feature space generated internally by the LLM to quantify the degree of matching between semantic features and visual fingerprint features.

[0115] It should be noted that VLM is used to extract semantic features from images and text, while LLM performs fast consistency calculation and matching based on pure text semantics. Therefore, this application uses LLM to process semantic features and visual fingerprint features, which can improve the overall processing efficiency.

[0116] For example, LLM constructs the model scoring criteria based on visual fingerprint features. After inputting the semantic features and visual fingerprint features corresponding to the street view images and panoramic map images of multiple candidate points into LLM, for each candidate point, the model performs a deep comparison of the semantic features of its street view image and panoramic map image with the visual fingerprint, and outputs a quantified matching score. Furthermore, the two optimal candidate points (first candidate point and second candidate point) can be selected in each road direction.

[0117] The matching score is a comprehensive metric that integrates information from multiple dimensions, including visual similarity, semantic consistency, and spatial layout rationality. A higher matching score indicates that the candidate point is more likely to be the device's current location.

[0118] Because intersections present multiple possible directions of travel, they are high-risk areas for misjudgment. Therefore, this application avoids positioning errors caused by directional misjudgment by listing and verifying all connecting roads. Furthermore, by utilizing the semantic and visual fingerprint features of candidate images from multiple candidate points, the application determines the first and second candidate points for each road, preventing the loss of potentially correct locations due to misjudgment of a single path and improving the fault tolerance in multi-directional intersection scenarios. In addition, by using composite candidates for comprehensive screening, the application enhances the overall discrimination capability for multi-road scenarios in complex intersection areas, improving positioning robustness.

[0119] Optionally, the target point is obtained by filtering the composite candidates corresponding to the target road, including:

[0120] The composite candidates corresponding to the connecting roads are compared and processed to determine the target candidate;

[0121] The candidate image and image data corresponding to the target candidate are matched to obtain the target point.

[0122] In this application, at intersection areas, target candidates are first determined by comparing composite candidates. Then, the best and second-best points contained in the target candidate are further matched to determine the target point. During the composite candidate comparison process, each composite candidate is semantically matched with visual fingerprint features. Competitive selection is performed based on the matching score, and the composite candidate with the highest score is retained as the target candidate.

[0123] Optionally, the target candidate can refer to the candidate that wins after a 1v1 elimination comparison among composite candidates, representing the set of candidate points with the best comprehensive semantic matching in the current intersection area. For example, the composite candidates are matched one-to-one to determine the target candidate. It should be noted that the specific process of comparison processing in this application embodiment is not limited. For example, it can also be a weighted comprehensive evaluation method, a similarity score comparison method, a voting elimination mechanism, etc.

[0124] Correspondingly, the target point can refer to the candidate point that is determined to be the best match for the image data after image matching. Optionally, the process of determining the target point can be carried out by inputting the candidate image corresponding to the target candidate and the image data into an artificial intelligence large model or a multimodal large language model for matching, or by calculating image similarity, or by inputting the semantic features of the candidate image corresponding to the target candidate and the semantic features of the image data into an LLM for matching scoring to determine the target point. The specific process of matching processing is not limited in the embodiments of this application.

[0125] For example, by comparing the composite candidates corresponding to each connecting road in pairs (using a 1v1 elimination mechanism), the optimal target candidate is determined through competitive screening; then the candidate images contained in the target candidate are matched and verified with the original image data to determine the target point.

[0126] In this way, by comparing all composite candidates corresponding to connecting roads, the most competitive candidate is selected, reducing redundant comparisons and improving decision-making efficiency in multi-candidate scenarios at intersections. Furthermore, by matching the candidate images and image data corresponding to the target candidate, the consistency between the target point and the original image data is ensured, enhancing the reliability of the localization results.

[0127] Optionally, in response to the initial coordinates being located in the road segment area, semantic consistency matching is performed between the semantic features of the candidate images and the visual fingerprint features to filter out the target point from multiple candidate points, including:

[0128] For the target road, identify multiple candidate points on the target road;

[0129] By utilizing the semantic features and visual fingerprint features of candidate images of multiple candidate points, semantic consistency is evaluated for each candidate point to determine the target point.

[0130] Optionally, starting from the initial coordinates, sampling is performed along the target road at preset distances in the first and second directions to obtain multiple candidate points; the first and second directions are opposite in direction.

[0131] The preset distance refers to the fixed interval between two adjacent candidate points when generating candidate points along the target road. This preset distance is a configurable parameter used to control the granularity of the search. A smaller preset distance can improve positioning accuracy but increase computational load; a larger preset distance can improve speed but may reduce accuracy. Its setting can be based on the application scenario requirements.

[0132] The first and second directions can refer to two opposite directions along the centerline of the current road segment. Typically, the first direction is defined as the direction of travel of the road, and the second direction is its opposite.

[0133] In this step, multiple candidate points can refer to a series of discrete location points generated by sampling along a first direction and a second direction on the target road at preset distance intervals. For example, these could be several location points sampled by a vehicle along the forward and reverse directions on the current road segment. The preset distance is less than or equal to a certain threshold, such as 200 meters.

[0134] In this way, by generating candidate points in both directions along the target road, it is possible to effectively cope with the uncertainty of the device's possible front and rear positions in the road segment, as well as the initial errors caused by GPS drift or ambiguity in direction judgment, thereby improving the fault tolerance and robustness of the positioning process.

[0135] For example, when the initial coordinates are located in a road segment area, multiple candidate points can be directly selected on the target road. Semantic consistency is evaluated by calculating the similarity between the semantic features and visual fingerprint features of each candidate point image, and the target point is directly determined based on the evaluation results. Semantic consistency evaluation refers to the process of quantifying the similarity between the semantic features and visual fingerprint features of candidate images in the semantic space and selecting the optimal candidate point based on this similarity score. This process can be implemented using LLM matching and scoring, as described in the above embodiments, and will not be repeated here.

[0136] Because road segment areas have a clearly defined linear structure, this application, given that the initial coordinates are located within a road segment area, can strictly constrain the search range to the target road and its adjacent extension directions. This avoids blind searching across the entire map, significantly improving search efficiency and computation speed, and achieving efficient directional search. Furthermore, target points are directly filtered through semantic consistency evaluation, maintaining the simplicity and specificity of the matching process. Therefore, the evaluation process for filtering target points in road segment scenarios is relatively simple; direct evaluation avoids redundant steps and improves positioning efficiency.

[0137] Understandably, this application designs a hybrid topology-aware search strategy that automatically switches between "intersection filtering" and "linear exploration" modes based on the initial coordinates to achieve dynamic adaptive positioning, thereby reducing the average number of search steps and improving the positioning success rate in complex urban environments.

[0138] For example, Figure 3 This is a flowchart illustrating an optional geospatial location positioning method provided in an embodiment of this application. Figure 4 This is a schematic diagram of the structure of a positioning system provided in an embodiment of this application, as shown below. Figure 3 and Figure 4 As shown, the positioning system includes a visual fingerprint generation module, a road network topology modeling module, an intelligent search scheduler, and a result generation module. The geospatial location positioning method may include the following steps:

[0139] Step 1: In response to the user's operation on the device, the user inputs initial coordinates, text data, and image data. After receiving the initial coordinates, text data, and image data, the visual fingerprint generation module extracts and fuses features from the initial coordinates, text data, and image data through a multimodal large language model to obtain visual fingerprint features, and then sends the visual fingerprint features to the processing module.

[0140] Step 2: The road network topology modeling module performs local road network topology modeling based on the initial coordinates, queries all surrounding local road network topology structures, that is, the set of road network nodes and connecting edges extracted within a limited space centered on the initial coordinates, further determines whether the initial coordinates are located in the intersection area or the road segment area, and sends the local road network topology structure to the processing module.

[0141] Step 3: The intelligent search scheduler activates the intelligent search strategy:

[0142] In one possible implementation, if a vehicle is determined to be located in an intersection area, an "intersection filtering" mode is initiated. Specifically, multiple connecting roads in the intersection area are identified as target roads, with an average of 3–6 roads involved per intersection. Furthermore, each connecting road is uniformly sampled, for example, 3 candidate points are sampled for each connecting road, and the semantic features of the candidate images corresponding to these 3 candidate points are obtained. LLM is used to process the semantic features and visual fingerprint features, that is, to evaluate the matching degree (matching score) between the semantic features and visual fingerprint features of the candidate images corresponding to these 3 candidate points, and the matching scores are batch sorted in one go to select the first candidate point and the second candidate point for each road. The images of the first candidate point and the second candidate point are packaged into a "composite candidate".

[0143] Furthermore, a 1v1 elimination tournament (simulating a tournament mechanism) is conducted among the composite candidates to determine the optimal point, i.e., the target point. Based on this target point, localization processing is performed, outputting the spatial localization result (target location) and a matching score. Optionally, this matching score can also assist in manual review of the localization results to improve credibility.

[0144] In another possible implementation, if the vehicle is determined to be located in a road segment area, a "linear exploration" mode is initiated. Specifically, multiple candidate points are generated at preset distances along the target road, either forward or backward. Furthermore, LLM is used to process the semantic features and visual fingerprint features of the candidate images corresponding to the multiple candidate points, that is, to evaluate the matching degree (matching score) between the semantic features and visual fingerprint features of the candidate images of the multiple candidate points, and to select the candidate point with the highest matching score as the target point.

[0145] Optionally, after obtaining the matching score, the matching score can be updated to the priority queue for use in the next iteration. In this way, when the vehicle is driving on a long straight road, since multiple candidate points need to be generated in real time, the matching score corresponding to the candidate point at the same position can be directly reused without using LLM for scoring again, ensuring efficient progress on long straight roads and avoiding invalid diffusion.

[0146] In this way, by leveraging the powerful understanding and reasoning capabilities of a multimodal large language model, deep cross-modal feature comparison is performed. Compared with traditional matching methods based on manual features or simple similarity, this avoids blind comparison and significantly improves the accuracy and semantic consistency of matching, achieving intelligent and high-precision matching. Furthermore, a composite verification step is constructed through pairwise comparison of "first candidate point and second candidate point." This effectively avoids misjudgments caused by a single candidate point accidentally having the highest score, improving the reliability and robustness of decision-making in similar roads or complex environments. Therefore, this application, through a hybrid topology-aware search strategy and a composite candidate comparison mechanism, solves the problems of low positioning accuracy and slow response in complex road networks of traditional methods, significantly improving positioning accuracy and robustness.

[0147] Optionally, determining the target location based on the target point includes: obtaining a spatial offset vector, and calculating the target location through coordinate transformation based on the target point and the spatial offset vector.

[0148] In this embodiment, the spatial offset vector quantifies the distance and direction of the coordinates deviating from the correct road centerline. Therefore, the spatial offset vector can be used to correct the target point, and a high-precision target position can be calculated through coordinate transformation.

[0149] The method for calculating the high-precision target position through coordinate transformation can refer to existing technologies or redefine new algorithms; this application does not specifically limit this method.

[0150] In this way, by applying spatial offset vectors, the positioning system can significantly compensate for coordinate errors, thereby obtaining a target position that is closer to the true position and effectively correcting positioning errors.

[0151] Optionally, the method is applied to autonomous vehicles, where the initial coordinates are the current initial position coordinates determined by a positioning device, and the image data are images captured by onboard sensors; the method further includes:

[0152] The target location is synchronized to the vehicle map to update the current position coordinates of the autonomous vehicle in the map coordinate system.

[0153] In this embodiment of the application, the vehicle map may refer to a map system installed in an autonomous vehicle that contains high-precision road information and semantic layers.

[0154] Map coordinate systems can refer to the unified geographic coordinate reference system used by in-vehicle maps to accurately describe the spatial location of vehicles and road features.

[0155] In this step, the determined target location is used as the precise location of the autonomous vehicle and synchronized to the onboard map system, enabling real-time updates of the vehicle's current position coordinates in the map coordinate system. This improves the accuracy of the autonomous vehicle's position perception by using the target location to correct initial coordinate errors. This achieves closed-loop integration of high-precision positioning and the map system, providing reliable position input for downstream tasks such as path planning and decision control.

[0156] In conjunction with the above embodiments, Figure 5 An interactive schematic diagram of a geospatial location positioning method provided in an embodiment of this application is shown below. Figure 5 The diagram illustrates the interaction between the user device and the positioning system. The positioning system may integrate a visual fingerprint generation module, a road network topology modeling module, a Visual Modeling Library (VLM), an intelligent search scheduler, a database engine, and a result generation module. Optionally, the VLM, intelligent search scheduler, and database engine can be deployed externally to the positioning system for use; this embodiment does not specifically limit this.

[0157] For example, a user device can submit multimodal data such as initial coordinate data, text data, and image data to a visual fingerprint generation module. The visual fingerprint generation module calls a Visual Model (VLM) to process the multimodal data and obtain visual fingerprint features. The VLM can return visual fingerprint features in JSON format to the visual fingerprint generation module.

[0158] The road network topology modeling module calls the database engine to query all surrounding local road network topology structures based on the initial coordinates. The database engine can return graph structure data with attributes from the road network topology modeling module. This graph structure data can be understood as the local road network topology structure and can be used to obtain subsequent target road and other related data.

[0159] The intelligent search scheduler enables intelligent search, which involves dynamic mode switching based on the initial coordinates. If the initial coordinates are located in an intersection area, a series of processing steps are performed, including batch sampling, matching and scoring, sorting, generating composite candidates, and a 1v1 elimination round comparison, returning the winning candidate. If the initial coordinates are located in a road segment area, a series of processing steps are performed, including linear expansion, matching and scoring, and updating the queue. It should be noted that the specific implementation methods corresponding to the above series of processing steps can be referred to the description in the above embodiment, and will not be repeated here.

[0160] The results generation module determines the target location based on the target point, and correspondingly, it can output the target location and matching score to the user's device.

[0161] Understandably, this application uses a simulated tournament mechanism to conduct a 1v1 elimination comparison of composite candidates, ultimately determining the best candidate, thus achieving a leap from "finding the point" to "confirming the point" and reaching the goal of "being able to see, compare accurately, and judge clearly".

[0162] Therefore, this application introduces Visual Modeling (VLM) into geospatial positioning, achieving a leap from passive execution to proactive planning, significantly improving positioning accuracy and exploration efficiency. It proposes using visual fingerprint features as a unified structured carrier for VLM input, mapping natural language and image semantics to a computable and comparable JSON standard format, solving the problem of quantifying unstructured inputs and significantly improving cross-modal matching accuracy. Furthermore, a hybrid topology-aware search strategy is designed, automatically switching between "intersection filtering" and "linear exploration" modes based on the current location, achieving dynamic adaptive positioning. This reduces the average number of search steps in complex urban environments and increases the positioning success rate. Moreover, the concept of "composite candidates" is introduced, constructing a two-level elimination-style comparison mechanism, making AI decision-making more adversarial and deterministic, and supporting highly reliable result output.

[0163] Optionally, the above methods can be applied to scenarios such as autonomous driving, drone navigation, and smart cities, achieving a 37% reduction in average positioning error, a 52% increase in path optimization rate, and enhanced robustness and generalization ability, providing a new paradigm for highly reliable and interpretable intelligent positioning. For example, in path planning, VLM is introduced for semantic understanding and direction prediction, overcoming the limitations of traditional methods. The algorithm overcomes the limitations of static weights by achieving dynamic perception and intelligent route selection, significantly reducing the misjudgment rate at complex intersections and improving path rationality. Simultaneously, through an adaptive step-size mechanism combined with image complexity analysis, the system can intelligently adjust the step-size according to the environment, overcoming the resource waste or target omission problems caused by fixed step-sizes. Furthermore, the dual-track parallel mechanism of deep exploration in the main task and broad coverage in the secondary task balances global perspective and local refinement, overcoming the tendency of a single exploration mode to get trapped in local optima. This reduces task completion time by 80% and increases the success rate to over 90%.

[0164] It should also be noted that the method proposed in this application systematically improves upon several limitations of existing technologies. Specifically, existing reinforcement learning-based exploration strategies, such as the use of Deep Q-Learning Network (DQN) based on deep learning, while capable of learning long-term rewards, have extremely high training costs, typically requiring tens of thousands of simulations to converge, making it difficult to meet the needs of real-time deployment; rule-based expert systems, while possessing good interpretability, face difficulties in rule maintenance, struggle to adapt to unknown scenarios, and have limited scalability; and while methods fusing Simultaneous Localization and Mapping (SLAM) and semantic segmentation improve environmental perception accuracy, they do not effectively address the "breadth-depth imbalance" problem at the exploration strategy level, still posing a risk of redundant exploration.

[0165] In the foregoing embodiments, the geospatial location positioning method provided by the embodiments of this application has been described. To implement the functions of the methods provided by the embodiments of this application, the electronic device serving as the execution subject may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0166] For example, Figure 6 This is a schematic diagram of the structure of a geospatial location positioning device provided in an embodiment of this application, as shown below. Figure 6 As shown, the geospatial location positioning device 600 includes:

[0167] The receiving module 601 is used to receive multimodal data, which includes initial coordinates, image data, and text data.

[0168] The input module 602 is used to input multimodal data into a multimodal large language model to obtain visual fingerprint features;

[0169] The first determining module 603 is used to determine the corresponding target road based on the initial coordinates;

[0170] The matching module 604 is used to acquire candidate images corresponding to multiple candidate points on the target road, and to perform semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to select the target point from the multiple candidate points.

[0171] The second determining module 605 is used to determine the target location based on the target point.

[0172] Optionally, the processing procedure of the multimodal large language model for multimodal data includes:

[0173] By using a multimodal large language model, road topology features corresponding to the initial coordinates, target features in image data, and semantic description features in text data are extracted. The road topology features, semantic description features, and target features are then fused to obtain visual fingerprint features.

[0174] Optionally, the first determining module 603 is specifically used for:

[0175] In response to the initial coordinates being located in the intersection area, multiple connecting roads in the intersection area are identified as target roads;

[0176] If the initial coordinates are located in a road segment area, then the road segment containing that area is determined as the target road.

[0177] Optionally, in response to the initial coordinates being located in the intersection area, matching module 604 is specifically used for:

[0178] For each connecting road, multiple candidate points for the connecting road are determined;

[0179] Using the semantic features and visual fingerprint features of candidate images of multiple candidate points, semantic consistency is evaluated and ranked for each candidate point to determine the first and second candidate points.

[0180] The candidate images of the first and second candidate points are packaged to generate composite candidates, and the composite candidates corresponding to the target road are filtered to obtain the target point.

[0181] Optionally, the matching module 604 includes a processing unit for:

[0182] All composite candidates corresponding to connecting roads are compared and processed to determine the target candidate;

[0183] The candidate image and image data corresponding to the target candidate are matched to obtain the target point.

[0184] Optionally, in response to the initial coordinates being located in the road segment area, matching module 604 is specifically used for:

[0185] For the target road, identify multiple candidate points on the target road;

[0186] By utilizing the semantic features and visual fingerprint features of candidate images of multiple candidate points, semantic consistency is evaluated for each candidate point to determine the target point.

[0187] Optionally, when applied to autonomous vehicles, the initial coordinates are the current initial position coordinates determined by the positioning device, and the image data are images captured by onboard sensors; the geospatial positioning device 600 also includes an update module, which is used for:

[0188] The target location is synchronized to the vehicle map to update the current position coordinates of the autonomous vehicle in the map coordinate system.

[0189] It should be noted that the specific implementation principle and effect of the above-mentioned geospatial location positioning device 600 can be found in the relevant descriptions and effects of the above embodiments, and will not be elaborated further here.

[0190] For example, Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device provided in the embodiments of this application may include: at least one processor 701; and a memory 702 communicatively connected to at least one processor; wherein the memory 702 stores instructions that can be executed by at least one processor 701, which are executed by at least one processor 701 to cause the electronic device to perform the method as described in any of the above embodiments.

[0191] Optionally, the memory 702 can be either standalone or integrated with the processor 701.

[0192] The memory 702 and the processor 701 can be connected via bus 703.

[0193] The implementation principle and technical effects of the electronic device provided in this application can be found in the foregoing embodiments, and will not be repeated here.

[0194] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the foregoing embodiments.

[0195] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the foregoing embodiments.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0197] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0198] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor. The memory may include RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0199] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0200] The aforementioned storage media can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0201] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0202] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0203] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0204] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0205] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A geospatial location positioning method, characterized in that, The method includes: Receive multimodal data, which includes initial coordinates, image data, and text data; The multimodal data is input into a multimodal large language model to obtain visual fingerprint features; Determine the corresponding target road based on the initial coordinates; Acquire candidate images corresponding to multiple candidate points on the target road, and perform semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to filter out the target points from the multiple candidate points; In response to the initial coordinates being located in a road segment area, the road segment where the road segment area is located is determined as the target road, and multiple candidate points on the target road are determined for the target road; Using the semantic features of the candidate images of the multiple candidate points and the visual fingerprint features, the semantic consistency of each candidate point is evaluated to determine the target point; The target location is determined based on the target point.

2. The method according to claim 1, characterized in that, The multimodal large language model's processing of the multimodal data includes: Using a multimodal large language model, the road topology features corresponding to the initial coordinates, the target features in the image data, and the semantic description features in the text data are extracted. The road topology features, the semantic description features, and the target features are then fused to obtain the visual fingerprint features.

3. The method according to claim 1, characterized in that, The method further includes: In response to the initial coordinates being located in an intersection area, multiple connecting roads in the intersection area are determined as target roads.

4. The method according to claim 3, characterized in that, In response to the initial coordinates being located in an intersection area, the step of performing semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to filter out the target point from the plurality of candidate points includes: For each connecting road, multiple candidate points for that connecting road are determined; Using the semantic features of the candidate images of the multiple candidate points and the visual fingerprint features, the semantic consistency of each candidate point is evaluated and ranked to determine the first candidate point and the second candidate point. The candidate images of the first candidate point and the second candidate point are packaged to generate composite candidates, and the composite candidates corresponding to the target road are filtered to obtain the target point.

5. The method according to claim 4, characterized in that, The process of filtering composite candidates corresponding to the target road to obtain the target point includes: All composite candidates corresponding to connecting roads are compared and processed to determine the target candidate; The candidate image corresponding to the target candidate is matched with the image data to obtain the target point.

6. The method according to claim 1, characterized in that, Applied to autonomous vehicles, the initial coordinates are the current initial position coordinates determined by a positioning device, and the image data are images captured by onboard sensors; the method further includes: The target location is synchronized to the vehicle map to update the current position coordinates of the autonomous vehicle in the map coordinate system.

7. A geospatial location positioning device, characterized in that, The device includes: A receiving module is used to receive multimodal data, which includes initial coordinates, image data, and text data; The input module is used to input the multimodal data into a multimodal large language model to obtain visual fingerprint features; The first determining module is used to determine the corresponding target road based on the initial coordinates; The matching module is used to acquire candidate images corresponding to multiple candidate points on the target road, and to perform semantic consistency matching between the semantic features of the candidate images and the visual fingerprint features to filter out the target points from the multiple candidate points. Wherein, in response to the initial coordinates being located in a road segment area, the road segment containing the road segment area is determined as the target road, and the matching module is specifically used for: For the target road, identify multiple candidate points on the target road; By utilizing the semantic features and visual fingerprint features of candidate images of multiple candidate points, semantic consistency is evaluated for each candidate point to determine the target point; The second determining module is used to determine the target location based on the target point.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.

Images