Identifying at least one geospatial location from user input

Abstract

A computer-implemented method of identifying at least one geospatial location according to user input is described. The method comprises: receiving (S100) a character string (10) from a user as the user input; using (S200) the string (10) for identifying at least one point of interest (POI); and identifying (S300) a geospatial location (50) corresponding to the at least one identified POI as the at least one geospatial location. Using (S200) the character string (10) includes querying a large language model (LLM) (40) based on at least a portion of the character string and a predetermined plurality of POI categories to find an output related to an identification of at least one POI category tag. Further, a method of updating POI information is provided, which can advantageously use the choices received by the user when performing the previous method.

Description

Technical Field

[0001] The present application relates to a computer-implemented method for identifying at least one geospatial location based on received user input. The present application further relates to a computer-implemented method for updating point of interest (POI) information (eg, as part of a POI database). Background Art

[0002] There are many ways to identify a geospatial location based on received user input, such as (but not exclusively) identifying a desired destination in the context of routing and navigation. When the user input corresponds to or contains an address, it is a simple matter of searching the electronic map to find this address. In the case where the address is incomplete or otherwise ambiguous, for example, if it contains only a street name and number, where this street name appears in various towns / cities, then multiple results may be presented. In order to rank these results, various factors (such as the current location (if known)) can be used to help the user quickly identify the desired geospatial location. Other potential factors that can be used (individually or in combination) to disambiguate the query and / or rank multiple potential results include the location of one or more previous searches and / or destinations; geographic areas known to be relevant to the user; relative popularity of the geographic area; etc. Similarly, if the user input corresponds to or contains the name of a point of interest (POI), such as the name of a business, then a POI database can be searched to find this name. For example, such a POI database can be a separate database linked to the electronic map, or can be part of a broader map database. Also here, in cases where multiple results correspond at least to some extent to the user input, multiple results may be presented, ranked using various latent factors (such as the latent factors mentioned previously).

[0003] However, user input is not always easy to disambiguate. This may be, for example, if the user does not remember the exact address or name; if the user input is incomplete and / or contains inaccuracies (e.g., spelling errors); if the user input resembles both a (partial) address and a POI name; and / or if the user is looking for some type of destination rather than a geospatial location corresponding to a specific place. Summary of the invention

[0004] In order to interpret a large number of user queries, a so-called Large Language Model (LLM) may be used. A Large Language Model is a probabilistic model of natural language that is trained on a text dataset comprising more than 500 million words, optionally more than 1 billion or 1 trillion words, and is adapted to generate language output based on a query (usually expressed as a prompt).

[0005] However, LLMs have several significant disadvantages. One is cost, both in terms of energy (and therefore cost) and time. This is due to the size of the model and the complexity of the processing. Another is accuracy, as the output of such models relies heavily on patterns in the training data, rather than reflecting an "understanding" of the ground truth. Not only do such models often return inaccurate results, or results that do not meet the parameters set by the query / prompt / instructions, but they are also prone to so-called "hallucinations", where the response is purely fictional.

[0006] Examples of methods according to the present disclosure can effectively utilize the functionality provided by the LLM to identify at least one geospatial location based on user input while minimizing the shortcomings inherent in the LLM. In particular, it is proposed to query the LLM using specific parameters and post-process the output. In some examples, it is further proposed to combine the LLM with other methods to interpret user intent to minimize the required cost while increasing or at least maintaining accuracy. Moreover, in some examples, it is proposed to treat the hallucinations of the model as features rather than vulnerabilities to update pre-existing POI databases and / or POI information.

[0007] In a first aspect, a computer-implemented method for identifying at least one geospatial location based on user input is provided. The method comprises receiving a string from a user as user input. This string may be received using a conventional input interface. For example, it may be a string typed into a search bar of a browser window, a navigation application, a navigation device, etc. It is also not excluded that the string is generated by a multi-step input process and / or a process in which the user is assisted (in an automated manner) in providing the string, such as using an auto-complete mechanism to suggest and / or provide words that are similar to and / or contain what the user has typed. It is also not excluded that receiving the string from the user already involves a certain degree of (pre-)processing, such as the string is generated by transcribing audio input.

[0008] The proposed method further includes using a string to identify at least one point of interest POI, and identifying a geospatial location corresponding to at least one identified POI as at least one geospatial location. It should be noted that in an embodiment, not all user inputs are intended to result in the identification of a POI. In particular, a user may simply want to find a specific address. Therefore, it may be considered to adopt a filtering mechanism to quickly identify a string corresponding to an address - for example, a simple heuristic may be used to determine whether a string contains any numbers, wherein a string containing numbers is considered to be likely to indicate and / or contain an address. In other embodiments, the search may be limited to a POI search (e.g., by user settings). In other embodiments, user input may be used to identify at least one POI and identify other locations (e.g., where the user input is unambiguously related only to a POI search). The embodiments of the present disclosure relate to POI searches, but do not exclude performing other searches (e.g., address searches) as described above.

[0009] Specifically, in the method of the first aspect, using the string includes querying a large language model LLM based on at least a portion of the string and a predetermined plurality of POI categories to find an output related to an identification of at least one POI category tag. It should be noted that this does not simply correspond to requiring the LLM to identify at least one geospatial location based on user input itself. Instead, it is proposed to use specific instructions to prompt the LLM to identify at least one POI category tag that is suitable for (or may be suitable for) at least a portion of the received string.

[0010] In an embodiment, querying the LLM for an output associated with an identification of at least one POI category marker based on at least a portion of the string may include providing a predetermined plurality of POI categories to the LLM.

[0011] Examples of POIs include gas stations, hotels, restaurants, supermarkets, etc. In the present disclosure, the term "POI category" is used for an abstract category associated with certain POIs, and "POI category tags" are generally used to identify terms of such categories. A "term" here may be a single word or a combination of several words (wherein a "word" may be an abbreviation / acronym). For a given POI category, there may be multiple POI category tags, for example including alternative spellings, synonyms and / or translations into other languages. Typically, one of the POI category tags for a certain POI category is selected from the POI category tags and used to refer to the POI category. In this description, for the purpose of readability, such selected terms or tags will be used when discussing the corresponding abstract POI category. For example, a POI category broadly defined as "a place where you can buy and drink coffee or tea" may have POI category tags "cafe", "coffee shop", "tea house", etc. Assuming that the selected POI category tag is "cafe", we will refer to it as "POI category 'cafe'" in this application. However, on database level, this may be appropriately implemented in a different way, e.g. where the POI categories are assigned some identifier that does not carry any linguistic meaning in itself. It should be noted that for a given POI category, not all POI category tags need to correspond to terms that can be used linguistically to identify such POIs: other POI category tags may contain terms that can be conceivably used as search terms by users searching for this POI category, such as "coffee", "tea", "latte", "cappuccino", etc. Some POIs may be associated with several categories; for example, a gas station may obviously be classified in the category "gas stations", but if it contains a shop, it may also be classified in the category of convenience stores. Depending on its offering, a venue may be classified in the POI categories "restaurant", "bar" and "café" at the same time. Furthermore, at least some of the POI categories may be associated with each other, e.g. in a hierarchical manner. As an example: the POI categories "hotel", "motel" and "bed and breakfast" may be considered as subcategories of the POI category "accommodation". Any POI associated with any of the POI categories "hotel", "motel" or "bed and breakfast" would then be associated (whether explicitly or implicitly) with the POI category "accommodation".

[0012] The POI category labeling may also include translating at least some of the POI category labels into at least one other language.

[0013] The foregoing, particularly the specific examples, are presented not to limit the present disclosure but rather to provide a context for understanding the description and to facilitate real-world explanation.

[0014] In an embodiment, providing the LLM with a predetermined plurality of POI categories may include providing at least one POI category tag for each POI category. For example, one POI category tag may be provided for each POI category. Additional POI category tags (including, for example, synonyms and / or translations) may be provided, but depending on the LLM, may not result in significantly improved performance because the LLM is typically already "aware" through its training whether two terms are commonly used to refer to the same thing. The predetermined plurality of POI categories may be provided as part of the prompt, or may be provided at an earlier point in time, for example, provided to the LLM in an earlier prompt or encoded into the LLM. Furthermore, in addition to the POI categories, at least one special tag (e.g., "empty", "not a category", etc.) may be provided as a possible output for the LLM, indicating that the portion of the string provided to the LLM is not considered to correspond to a POI category. Such special tags may be used (alternatively or additionally) when the LLM is unable to provide a category tag. Furthermore, it should be noted that the predetermined plurality of POI categories may be updated over time (as may the associated plurality of POI category tags). One possible way of doing this will be described below through the second aspect. Such updated multiple POI categories may be provided to the LLM.

[0015] In an embodiment of the method of the first aspect, using the string may include extracting at least one candidate POI category tag from the output of the LLM; and identifying the at least one POI based on the at least one candidate POI category tag. Since LLM models are usually designed to communicate with users through natural language, the output of such models may be very lengthy and contain irrelevant text, such as introducing answers ("The category that most closely corresponds to your input is") or suggesting additional steps ("Do you want to know more?"). Therefore, extracting at least one candidate POI category tag from the output of the LLM may include separating at least one word or group of words from the output that is considered to be likely to correspond to a category tag. This may be done by removing commonly used irrelevant text; by filtering out words that are unlikely to contain meaning (such as articles and pronouns); by searching the output of POI category tags; by performing another LLM query to narrow the output, etc. It does not exclude that several candidate POI category tags may be extracted, nor does it exclude that some of these candidate POI category tags may overlap to some extent.

[0016] In an embodiment, identifying the at least one POI based on the at least one candidate POI category tag may include determining whether the at least one candidate POI category tag corresponds to at least one POI category of the predetermined plurality of POI categories. For example, each of the at least one candidate POI category tag may be compared to a list of existing POI category tags, wherein each POI category tag is associated with an existing POI category. The list may include at least one POI category tag for each of the predetermined plurality of POI categories, but advantageously includes additional POI category tags for at least some of the POI categories. In particular, it should be noted that if the LLM is provided with a predetermined plurality of POI categories as described above, then the list of existing POI category tags need not correspond one-to-one to what is provided to the LLM. After all, the LLM may appropriately respond to the prompt “Is this a coffee shop?” with the answer “Yes, this is a coffee shop.” At least if the candidate POI category tag is an exact match to an existing POI category tag, then the candidate POI category tag is said to correspond to the existing POI category tag (and hence the associated POI category); however, it should be noted that even if there is not an exact match, for example if there is a slight difference in spelling ("harbor" vs. "harbour") or font ("café" vs. "cafe"), the candidate POI category tag may be determined to correspond to the existing POI category tag. This avoids having to include POI category tags for all possible variations of the term in the list of POI category tags, and may further take into account the fact that many current LLMs are known to deviate from instructions, and may appropriately respond to a prompt of "is this green or gray" with "is this gray" (using American English rather than British English). Known techniques for "fuzzy searching" may be applied to assess whether the extracted candidate POI category tag corresponds to an existing POI category tag. Alternatively or additionally, in an example, a subsequent query may be performed to clarify the output.

[0017] Optionally, the method further comprises: in response to determining that the at least one candidate POI category tag corresponds to at least one of the predetermined plurality of POI categories, identifying the at least one POI based on at least one corresponding POI category of the predetermined plurality of POI categories. For example, if the candidate POI category tag corresponds to an existing POI category tag, then a POI database may be searched to find at least one POI associated with the POI category corresponding to the POI category tag. In most cases, there will be several POIs associated with a POI category. Different results may be ranked based on various factors. Examples include (but are not limited to): (distance) from the current location, (distance) from at least one previously searched location; (distance) from a user-specified location, relative relevance / popularity of a geographic area in which the POI is located; user preferences; etc. In addition, portions of the string received from the user may be considered to filter and / or rank the results. For example, if the string also includes the name of a town or city, then POIs in this town or city may be prioritized.

[0018] In several cases, the output of the LLM will at least largely conform to the instructions: for example, an LLM instructed to identify whether "X" corresponds to POI category "A", "B", or "C" provides output corresponding to or containing "A".

[0019] However, as mentioned, LLMs are not always "obedient" to such an extent, and in several cases, the output may correspond to "D" - in the context of LLMs, this is often referred to as a hallucination. Typically, such hallucinations are considered errors and are discarded.

[0020] However, in an example, the method advantageously further comprises, in response to determining that the at least one candidate POI category label does not correspond to at least one POI category of the predetermined plurality of POI categories, identifying the at least one POI by searching a POI database and / or a map database based on the candidate POI category label.

[0021] In some cases, hallucinations may indicate gaps or incompleteness in the POI database, but may be able to lead to relevant results. For example, let's take an example where a user searches for "Picasso". In this example, this may not be an existing POI category tag or a keyword corresponding to a POI category, so a simple search of the prior art would only produce results such as streets named after the artist. Based on "Picasso", querying the large language model LLM to find outputs related to identifying at least one POI category tag may produce "museum". "Museum" is likely to be an existing POI category tag. However, "art" may also be returned. In this example, this is not an existing POI category tag, but relevant results can still be found by searching the POI database and / or map database based on the term "art" because the word is likely to appear in the names and / or descriptions of museums as well as galleries, art schools, etc. It should be noted that such a search can be done in addition to and / or in conjunction with searching the POI database and / or map database based on the original search string (here "Picasso").

[0022] In view of the cost and computational workload involved in each LLM query, the database may maintain storage of combinations of LLM inputs and outputs (of POI candidate labels extracted therefrom), i.e., here, "Picasso" and "Art". If the query "Picasso" is received again, then it may be considered to retrieve the output "Art" from the database, rather than querying the LLM again. Therefore, the embodiments of the first aspect may achieve a reduction in computational workload and a corresponding reduction in energy consumption. It should be noted that since the LLM is non-deterministic (that is, the same input does not necessarily lead to the same output) - it may also be considered to query the LLM up to n times (n is a predetermined integer) using the same input, and store each resulting candidate POI category label for later use.

[0023] The method may further comprise providing for displaying data of at least one identified POI to a user in a selectable manner, optionally in combination and / or grouped together with a corresponding candidate POI category tag. There are various ways to do this on a graphical user interface. For example, the candidate POI category tag used to identify the at least one POI may be displayed in a manner that visually distinguishes it from tags of existing POI categories used in other cases, for example by adopting different colors, transparencies, groupings, labels and / or styles. Thus, embodiments of the first aspect may enable an objectively better way of presenting search results to a user, because the user may identify the search results based on the candidate POI category tag, even in cases where the data in the POI database does not include the corresponding POI category tag, enabling the user to objectively better envision the information presented.

[0024] The method may further include, in response to receiving a selection of one of the at least one POI identified by searching the POI database and / or the map database based on the candidate POI category tag, storing data indicating an association between the selected POI and the corresponding candidate POI category tag, for example in a data storage device. The selection of such a POI may be an indication that the particular POI is indeed related to the original string received from the user, particularly in the case where it is displayed in a different manner with the geospatial location derived from the user query.

[0025] In response to determining that at least one candidate POI category marker does not correspond to at least one POI category of a predetermined plurality of POI categories, the method may include providing data for displaying at least one candidate POI category marker to a user in a selectable manner. Then, the method optionally further includes, in response to receiving a selection of at least one displayed candidate POI category marker, providing at least one POI corresponding to the selected candidate POI category marker. In this case, it may be considered to add the candidate POI category marker to the existing POI category marker (directly, or later based on statistical analysis of the aggregated results). Therefore, embodiments of the first aspect may enable supplementing existing POI information with additional data points; correspondingly, the POI information may be maintained at least partially in an automatic manner.

[0026] In an embodiment, there may be situations where identification of suitable POIs does not require a LLM.

[0027] For example, in an embodiment, using the string for identifying at least one POI may further comprise determining whether at least part of the string satisfies matching criteria with at least one of a predetermined plurality of POI categories, wherein optionally each POI category is associated with at least one POI category tag.

[0028] In an embodiment, querying the LLM to find an output associated with the identification of at least one POI category tag may only be performed in response to determining that the string does not satisfy the matching criteria. Thus, in an embodiment, querying the LLM may further assist in minimizing computational effort, and therefore in particular energy consumption, when simpler (and therefore more computationally efficient) methods do not produce results.

[0029] In addition, in response to determining that at least a portion of the string satisfies the matching criteria for at least one POI category, the method may include identifying at least one POI based on the at least one POI category. In this way, where possible, existing category tags may be advantageously used, whether instead of or in addition to using the LLM. It should be noted that many techniques for evaluation of matching criteria will be faster and / or more computationally efficient (and therefore less expensive) than querying the LLM. Therefore, it may be advantageous to use matching criteria to determine in a fast, efficient manner whether the string can be processed / interpreted without the LLM, or whether the string is more difficult to disambiguate.

[0030] To illustrate the above, the following table shows the response time and accuracy for a number of different techniques when tested on a set of 25 intentionally ambiguous (English) queries, meaning that the tokens in these queries do not match any existing POI categories or POI category tags:

[0031] technology Response time Accuracy Simple matching 128ms 8％ Fuzzy matching 410ms 24％ GPT4-32k 2546ms 80％ GPT 3.5Turbo 853ms 84％ Vector Indexing 85ms 36％

[0032] Determining whether at least a portion of a character string satisfies matching criteria with at least one of a predetermined plurality of POI categories may be accomplished in a variety of ways (either independently or in combination).

[0033] For example, determining whether at least a portion of the string satisfies a matching criterion with at least one of a plurality of predetermined POI categories may include, for each POI category tag, identifying whether there is a corresponding relationship between the POI category tag and at least a portion of the string. The matching criterion may then be whether there is a corresponding relationship between at least a portion of the string and at least one POI category tag. It should be noted that while a "corresponding relationship" may be considered to exist when there is an exact match, this is not required in all cases (and in fact is not usually required). As described in the context of determining whether at least one candidate POI category tag corresponds to at least one POI category of a plurality of predetermined POI categories, various "fuzzy search" methods may also be employed herein.

[0034] Additionally or alternatively, determining whether at least part of the string satisfies a matching criterion with at least one of the predetermined plurality of POI categories may include calculating, for each of the predetermined plurality of POI categories, a similarity parameter between the string and the POI category. The matching criterion may then be whether the similarity parameter exceeds a threshold for at least one POI category.

[0035] Additionally or alternatively, determining whether at least part of the string satisfies a matching criterion with at least one of the predetermined plurality of POI categories may include converting at least part of the received string into a search vector, calculating a distance between the search vector and a vector representation of the predetermined plurality of POI categories. This is a known way of evaluating semantic similarity. The matching criterion may then be determined based on the number of POI categories for which the distance does not exceed a threshold.

[0036] In cases where several of the described techniques for evaluating matching criteria are used in combination, they may be performed simultaneously and / or in parallel. However, they may also be performed in at least a partially hierarchical and / or dependent manner. For example, one may consider first evaluating whether there is an exact match to an existing POI category tag; a "fuzzy match" may be evaluated next or simultaneously, but may also be evaluated only if it is determined that an exact match does not exist. Vector indexing, which is generally a slower and / or more computationally intensive technique than simple matching (although still generally faster and / or less computationally intensive than LLM queries), may be performed next or in parallel, or only if an initial match fails. Other combinations and / or dependencies may be considered to achieve a desired balance between speed, accuracy, and completeness.

[0037] It should be noted that for any or all of the matching techniques, the matching criteria may be evaluated for a single POI category tag for each POI category or for several / all POI category tags associated with a POI category. Similar or different thresholds (thresholds or threshold values) for different matches or matching techniques may apply.

[0038] In an embodiment, using the string may further include, in response to the output from the LLM indicating failure of identification of at least one candidate POI category label, identifying at least one POI by searching a POI database and / or a map database based on at least a portion of the string.

[0039] Even in the case where some filtering occurs to identify queries that may not indicate a POI category, the string provided to the LLM to find an identification of at least one POI category label may not contain and / or indicate a possible POI category, but may instead be associated with an address or with a POI name, for example. The LLM may also simply be unable to identify a suitable candidate POI category label. In such a case, the output may not be meaningful or usable. In embodiments, it may be advantageous to identify situations in which the LLM output does not produce meaningful results.

[0040] One possibility for determining whether the output from the LLM indicates a failure in identification of at least one candidate POI category tag may be to check whether the output corresponds to and / or contains a word, and potentially to perform some type of semantic analysis, for example by checking whether extraction of at least one candidate POI category tag from the output of the LLM has failed. Alternatively or in addition, instructions may be provided to the LLM to indicate a failure in identification of at least one POI category tag in a particular manner. This may be done together with instructions to identify the candidate POI category tags, and / or may be done using some type of previous instructions / coding. The indication may involve the use of a certain term (e.g. "not a category"), a code ("404"), or any alternative solution that allows for simple identification of such output (e.g. failure). For example, the notation "not a category" may be provided to the LLM together with a plurality of predetermined plurality of POI categories, for example as part of a prompt.

[0041] In response to the output from the LLM indicating failure of identification of at least one candidate POI category label, at least one POI may still be identified by searching a POI database and / or a map database based on at least a portion of the original search query. For example, at least a portion of the string may correspond to a POI name.

[0042] It should be noted that even in the case where at least one POI is successfully identified based on the output of the LLM, in embodiments, using the string further includes identifying at least one other POI based on at least a portion of the string, for example by searching a POI database and / or a map database based on at least a portion of the string. The method may include providing data for displaying at least one geospatial location to a user. Optionally, the method may further include displaying at least one geospatial location to a user based on the provided data. It should be noted that in embodiments, this will include displaying multiple geospatial locations for the user to select, as well as other elements that allow the user to further refine the results. Such multiple geospatial locations may include geospatial locations identified using different techniques from those described herein, including (but not limited to): querying the LLM to find output associated with the identification of at least one candidate POI category label and identifying at least one POI based on at least one associated candidate POI category; querying the LLM to find output associated with the identification of at least one candidate POI category label and searching a POI database and / or a map database based on at least one candidate POI category label; determining whether at least a portion of a string satisfies matching criteria with at least one of a predetermined plurality of POI categories and identifying at least one POI based on at least one POI category satisfying the matching criteria; and searching a POI database and / or a map database based on at least a portion of the string. In addition, for some or all of the geospatial locations, the data provided for display may include an indication of its source, i.e., the technology and / or terminology that resulted in the identification of a particular geospatial location.

[0043] Providing data for displaying at least one geospatial location to a user may include providing data for visually indicating, for at least one of the geospatial locations, that the geospatial location is based on a POI category identification. Visually indicating that the geospatial location is based on a POI category identification may include visually indicating, for each geospatial location based on the POI category identification, a visual indication of the POI category. This may be done by displaying an indicative mark of the POI category, using color coding and / or other visual coding, and / or using symbols that indicate the nature of the POI category (e.g., a knife and fork for indicating a "restaurant", a coffee cup for indicating a "cafe", a banknote for indicating an "ATM", etc.). Furthermore, in an embodiment, the predetermined POI category (or POI category mark) may be visually distinguished from candidate POI category marks that do not correspond to one of the predetermined plurality of POI categories or corresponding POI category marks, because the latter are more speculative and are not (yet) associated with the POI as a POI category in the POI database. For example, color coding may be used for the term itself or for a GUI element that displays the term.

[0044] Alternatively or additionally, when a plurality of geospatial locations are identified (in this case), providing data for displaying at least one geospatial location to the user may include providing data for grouping and / or ranking the plurality of geospatial locations according to corresponding POI categories.

[0045] Alternatively or in addition, the method may include providing data for displaying a visual indication of at least one POI category and / or candidate POI category mark identified using a string in a selectable manner, and in response to receiving a selection of a POI category or candidate POI category mark, identifying at least one geospatial location based on the selection and providing data for displaying the at least one geospatial location. Optionally, the method may further include displaying a visual indication in a selectable manner based on the provided data. This may provide a way to "verify" the identified POI category (or candidate POI category mark) before identifying the corresponding geospatial location. It should be noted that identifying at least one geospatial location based on at least one POI category and / or candidate POI category mark identified using a string may have been performed or at least partially performed before receiving a selection of a POI category or candidate POI category mark. In response to receiving a selection of a POI category or candidate POI category mark from a user, data is provided for displaying at least one geospatial location identified based on the selection. In this way, response time may be reduced. It may be advantageous to create a visual distinction (e.g., via color coding) between the way "existing" POI categories and candidate POI category marks (which do not clearly correspond to any existing POI category) may be displayed.

[0046] Alternatively or in addition, when a plurality of geospatial locations are identified, providing data to a user for displaying the plurality of geospatial locations may further include providing data for selectably displaying at least one POI category corresponding to at least one of the plurality of geospatial locations and / or a candidate POI category indicia identifying the at least one geospatial location, wherein optionally, in response to receiving a selection of a POI category or a candidate POI category indicia, the method further includes filtering and / or re-ranking the plurality of geospatial locations based on the selected POI category or candidate POI category indicia. The POI categories may be displayed in a variety of ways, including using symbols and / or color coding, and / or a visual distinction may be made between the display of "existing" POI categories and candidate POI category indicia (which do not explicitly correspond to any existing POI category).

[0047] It should be noted that where the results to be displayed are obtained using at least two different techniques, in embodiments, statistics about the selections made by the user may be compiled, and optionally such statistics may be used to rank / filter / sort subsequent results. This may be done globally and / or in a user-specific manner based on user-specific selections, and / or for groups of users that have been grouped together based on, for example, search behavior and / or other characteristics.

[0048] The method may include providing data for displaying the at least one identified geospatial location to a user in a selectable manner, and optionally, displaying the at least one identified geospatial location to the user in a selectable manner based on the provided data. The method may further include, in response to receiving a selection of a geospatial location, providing data indicating instructions for navigating to the selected geospatial location. Such instructions may include routing and guidance instructions for a driver of a vehicle or a pedestrian, and such instructions may also include routing instructions for an automated and / or assisted driving system. The method may also include displaying instructions to a driver of a vehicle or a pedestrian, for example via a display screen, and / or providing instructions to an automated and / or assisted driving system.

[0049] In a second aspect, the present disclosure relates to a method for updating point of interest (POI) information. The method may use a POI database, wherein a plurality of POIs are associated with at least one of a predetermined plurality of POI categories. Such a POI database may also include at least part of the POI information. The POI information includes at least one POI category tag associated with each POI category. The method includes accessing a data storage device, such as that obtainable by the method described in the context of the first aspect. The method further includes, for a given candidate POI category tag, determining the frequency of occurrence in the POI database of at least one of the predetermined plurality of POI categories associated with a POI having an association with the candidate POI category tag stored in the data storage device. In response to determining that the frequency of occurrence meets a first criterion, the method includes adding the candidate POI category tag as a POI category tag associated with at least one of the predetermined plurality of POI categories. Furthermore, in response to determining that the frequency of occurrence satisfies a second criterion, the method comprises adding the candidate POI category label as a new POI category to a database, in particular a POI category label corresponding to the new POI category, and associating the new POI category with POIs having an association with the candidate POI category label stored in the data storage device.

[0050] This second aspect is based on the fact that the illusion of LLM, or the insight that at least some of the outputs that do not strictly satisfy the instructions, can be advantageously used to enrich / update pre-existing databases, thus converting vulnerabilities into features.

[0051] In an embodiment, a data storage device as previously described in the context of the first aspect is used, particularly in the case where a user has selected a POI discovered based on a candidate POI category label from the output of the LLM that does not correspond to a pre-existing POI category label, a data storage device storing at least the selected POI and the corresponding candidate POI category label. Other details may also be stored, particularly the original search string.

[0052] By gradually accumulating and analyzing data about user responses to POIs identified based on LLM output and updating the POI database (or other POI information), new tags may be identified and new categories may be discovered. Over time, particularly if used in conjunction with a log of LLM input and output (e.g., as described above), this may result in an increased percentage of character strings resulting in the identification of POIs that are likely to be selected by users without requiring further use of the LLM. In this way, the LLM may help reduce its own necessity and may increase the relevance of search results, even for future user queries.

[0053] The methods of the first and second aspects are computer-implemented methods, which can be implemented on different types of computing systems, in particular the system of the fourth aspect.

[0054] In a third aspect, a computer program product is provided, comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to the first and / or second aspects. Such a computer program product may comprise the computer-readable non-transitory storage medium of the fifth aspect.

[0055] In a fourth aspect, a system is provided, comprising at least one integrated circuit, the system being configured to perform the method according to the first and / or second aspect.

[0056] The system may be a server, such as a business information server and / or a distributed computing system (e.g., a computer cloud). In an embodiment, a computing system may include a client and a server, wherein method elements that display data or provide instructions may be performed at the client, and other method elements are performed at the server. The client may be a mobile device, such as a smartphone, an in-vehicle computing device, etc. The server itself may be a business server and / or a distributed computing environment.

[0057] In a fifth aspect, there is provided a computer-readable non-transitory storage medium having instructions stored thereon which, when executed by a system comprising at least one integrated circuit, in particular the system of the fourth aspect, cause the system to perform a method according to the first and / or second aspects. Such computer-readable non-transitory storage media may include hard disks, Internet / cloud storage, memory drives, CD-ROMs, and any type of computing system that includes the capability to store data in a computer-readable non-transitory manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further illustrated by the accompanying drawings, in which:

[0059] Figure 1 is a block diagram illustrating various elements that may be used to perform methods according to the present disclosure;

[0060] Figure 2 , 3 , 4 and 5 are flowcharts illustrating various examples according to the first aspect;

[0061] Figure 6 is a flow chart illustrating an example according to the second aspect;

[0062] Figure 7 is a system diagram showing elements of an exemplary system according to the present disclosure;

[0063] Figure 8 show example screens that may be displayed in the context of the proposed method,

[0064] Fig.9A and 9B Displays other instance screens for the same user input, where Fig. 9B Describe the possible improvements based on the use of LLM. DETAILED DESCRIPTION

[0065] It should be noted that throughout the accompanying drawings, similar numbers are used for similar steps or elements, and the descriptions of these steps or elements will not always be repeated; the description of a step or element in the context of one figure can be considered applicable to other figures containing this step or element unless otherwise stated.

[0066] Figure 1 is a block diagram illustrating various elements that may be used to perform methods according to the present disclosure.

[0067] The method described herein may be intended to identify at least one geospatial location based on user input, for example in the context of a navigation input. For example, such user input may be received from a vehicle 11, specifically a navigation system in the vehicle, but may also be received on a smartphone 12 or other wireless personal device, or from yet another source. The present disclosure is not limited to any particular source, nor to a particular input method, as long as a string 10 is received as user input directly or indirectly from a user in step S100. Here, the string may be interpreted as a certain number of characters, so that the string has a sufficient length to give it a meaning with at least a certain probability. For example, it may contain at least one word, but the proposed method is configured so that in some cases, a successful result can also be achieved if the string contains only partial and / or misspelled words. The string may also (in addition or alternatively) include other characters, such as numbers.

[0068] The string 10 is then processed by a processing module 100. The processing module 100, which may be embodied on various types of computing systems, may be implemented on the same device that receives the user input, such as a vehicle 11 or a smartphone 12, in which case the string 10 is received directly from an input member on this device with which the user interacts. The processing module 100 may also (alternatively or additionally) be implemented on a separate device, which then receives the string 10 via some communication channel at step S100. It is also contemplated that certain steps are performed on the device that receives the user input, and other steps are performed on a separate device.

[0069] At S200, the string 10 is used to identify at least one point of interest or POI. To this end, the processing module interacts with a POI database 21 (also referred to elsewhere in the figure as POIDB for efficiency). In the figure, the POI database 21 is depicted as part of a map database 20; however, the POI database may also be a separate database, optionally linked to the map database. The POI database (or at least one copy of the POI database) may be stored on the same device on which the processing module 100 is implemented, but this is not required as long as the processing module 100 can retrieve data from the POI database 21.

[0070] S100 may include (pre)processing the string 10. For example, it may be evaluated whether the string 10 contains the name of a city. In this case, the name of the city may be filtered out, and / or the name of the city may be used at a later stage to filter / select multiple POIs.

[0071] S200 includes, at least for some user inputs, querying a large language model 40 based on at least a portion of the string 10 and a predefined plurality of POI categories 22 to find an output related to the identification of at least one POI category label. While it is not excluded that the LLM may be optimized at some point such that it can be (at least partially) implemented on a personal device (such as a vehicle 11 or a smartphone 12), the significant computational power required for such queries generally requires the LLM 40 to be implemented on a remote server. The processing module 100 may be implemented on this server. The processing module may also interact with the LLM 40 by sending a query (a so-called "prompt") to the LLM 40 and receiving the LLM output as a response. This "prompt" here generally includes not only at least a portion of the string 10 but also an indication to seek an output related to the identification of at least one POI category label. Specifically, the query of the LLM 40 is based on at least a portion of the string 10 and a predefined plurality of POI categories 22. This predefined plurality of POI categories 22 preferably includes the POI categories that occur in the POI database 21. The POI categories 22 may be derived from the POI database 21 and may optionally be stored as part of the POI database 21. Storing the POI categories 22 on the same device on which the processing module 100 is implemented may be efficient as the required storage capacity may be limited. Each POI category in the predefined plurality of POI categories 22 may be associated with at least one POI category label in the POI category labels 23, as previously described.

[0072] Querying the LLM specifically may help to disambiguate more difficult and / or more exotic queries. It may further help to process user queries in less frequently used languages that are (not yet) considered in another way, such as by including translations in the list of POI category labels 23.

[0073] S200 results in the identification of at least one POI. It should be noted that other factors may also be used in the identification of at least one POI, in particular selecting from a plurality of POIs identified using a character string the at least one POI estimated to correspond to the user input. Factors may be the user's current location; previous user behavior; the current displayed section of the map; the relative popularity of the POIs, etc. If more than one POI is identified, such factors may also (in addition or alternatively) be used to rank these POIs.

[0074] Other details about S200 will be explained through other drawings.

[0075] At S300, a geospatial location 50 corresponding to at least one identified POI is identified as at least one geospatial location. In particular, at least one address and / or latitude / longitude for the identified POI may be retrieved from the map database 20. At this step, other geospatial locations may also be identified (in addition or alternatively).

[0076] Figure 2 is a flow chart illustrating possible steps involved in querying the LLM 40. In particular, at S230, the LLM 40 is queried to find output associated with the identification of at least one POI category tag. To this end, the LLM may be provided with a predetermined plurality of POI categories 22, optionally including POI category tags 23. The LLM 40 may be provided with the POI categories 22 and optionally the category tags 23 separately. The POI categories 22 and optionally the category tags 23 may have been provided ( / hard-coded) at some earlier stage. In addition, although Figure 2 An arrow is shown from the POI category 22 to the LLM 40 , but the POI category 22 may be provided as part of the prompt sent to the LLM 40 as part of the query of S230 .

[0077] In response to the query / prompt, the LLM 40 provides an output. This output may include at least one POI category label corresponding to one (or more) of the predetermined plurality of POI categories 22, but due to the unpredictability of the LLM, this is uncertain. Furthermore, it may include additional elements.

[0078] At S235, at least one candidate POI category label is extracted from the output. This can be done in various ways, such as by filtering out common introductory and / or concluding terms and phrases.

[0079] At S250 , the extracted candidate POI category tags are used to identify at least one POI. Figure 3 The flowchart of provides more details on how S250 may be implemented.

[0080] Specifically, at S240, it can be evaluated whether the extracted candidate POI category label (or each of the extracted candidate POI category labels, provided there are more than one) corresponds to a known category. This can be done by comparing the extracted POI category with POI category 22 and / or POI category label 23. It should be noted that even if the LLM 40 is only provided with POI category 22 and does not have POI category label 23, the comparison with POI category label 23 may be useful because the LLM 40 can output terms not explicitly indicated in the query or prompt, in addition to synonyms and / or language variants of such terms.

[0081] The extracted POI category label can be said to correspond to an existing POI category and / or category label if, upon examination by comparing the extracted candidate category label or each of the extracted candidate category labels against a predetermined plurality of POI categories (preferably including associated labels), the extracted POI category label exactly matches. This is computationally the most efficient, but may not fully account for the fact that the LLM output may not exactly correspond to one of the possibilities in the prompt or otherwise provided beforehand, for example because semantic context may result in output grammar variants ("restaurants" instead of "restaurant"), because different spelling conventions may be used ("harbor" vs. "harbour"), or because the LLM deviates from the parameters of the prompt for some other reason. To account for this, some form of "fuzzy" matching can also be employed, such that such approximate matches can also be used to indicate correspondence. Semantic similarity measurements, as described later, can also be considered - in each case, preferably using a method that optimally balances the computational efficiency and accuracy of a particular implementation. This may also require further subdivision of the evaluation, for example by first evaluating exact matches and only attempting other matches if no exact match is found. Additionally, it should be noted that the evaluation at S240 is not necessarily binary: similarity criteria can be used to indicate the degree of correspondence between the extracted POI category label and an existing POI category, and this similarity criteria can be output together with the (at least one) POI category that is considered likely to correspond to the extracted POI category label. Such a degree of correspondence can be used, for example, to filter, organize, and / or rank the POIs and / or corresponding geospatial locations that are ultimately identified as being linked to a plurality of POI categories that are found to have at least some degree of correspondence with the extracted candidate POI category label.

[0082] If it is determined that at least one of the extracted POI category tags corresponds to an existing POI category in the predetermined plurality of POI categories 22, or has a likelihood of corresponding to an existing POI category in the predetermined plurality of POI categories 22 exceeding a certain threshold, then S250 may include identifying at least one POI associated with this POI category using the POI database 21 at S251. Since most POI categories may be associated with many POIs, various factors may be used to filter / select at least one POI. This includes the factors discussed previously. Additionally or alternatively, a portion of the string 10 initially input by the user may be evaluated to determine whether it includes information based on which at least one POI may be selected. For example, if the string 10 includes the name of a city, then POIs in this city may be identified and / or given a priority ranking. Additionally or alternatively, other elements of the LLM output may be used to determine the selection and / or ranking of POIs.

[0083] Using the extracted candidate POI category tags to identify at least one POI, S250 may include searching the POI database 21 and / or the map database 20 based on the extracted candidate category tags at S252. Figure 3 This is shown as occurring only when the (or each) extracted candidate category label is not a known category, but this is not the case in every implementation. For example, in a case where there are multiple extracted candidate category labels, only some of which are found to correspond to existing POI categories at S240, the POI database 21 and / or map database 20 may be searched based on at least one of the extracted candidate category labels not being found to correspond to an existing POI category.

[0084] Additionally or alternatively, in the case where S240 involves evaluating a similarity criterion, two thresholds may be considered: if the extracted candidate category tag has a similarity criterion exceeding a higher threshold, then at least one POI associated with the corresponding existing POI category may be identified; if the similarity criterion is between the lower and upper thresholds, then the POI may be identified based on the corresponding existing POI category and by searching the POI database 21 and / or the map database 20 based on the candidate category tags; and if the similarity criterion is below the lower threshold, then only a search based on the candidate category tags may be performed.

[0085] It is also possible to consider searching the POI database 21 and / or the map database 20 based on the (or each) candidate category marker, regardless of whether it is found to correspond to a known category marker (and / or how close it is to a known category marker), which can enrich the search results and help update the database as will be further described.

[0086] Optionally, using the string 10 for identifying at least one point of interest or POI, S200 further comprises determining at S245 whether the output from the LLM indicates a failure in the identification of at least one candidate POI category label. It should be noted that although if it is determined at S240 that the (or each) extracted candidate POI category label does not correspond to a known POI category (and / or POI category label), then Figure 3 Determining failure is shown as occurring, but this may not necessarily be the case in all embodiments. For example, in the event that the LLM itself determines that it was unable to successfully identify a candidate POI category tag, the LLM may be provided with a specific "tag" or description to use, such as "not a category", "404", etc. In this case, evaluating whether the (or any) extracted candidate POI category tag corresponds to a known POI category (and / or POI category tag) may include evaluating whether the (or any) extracted candidate POI category tag corresponds to this "failure" tag or description. This may be done instead of or in addition to a separate failure evaluation (which may involve more complex analysis of the extracted candidate POI category tags) that may occur in the event that the extracted candidate POI category tags at S240 do not correspond to a known category (or failure tag). Additionally or alternatively, the failure to determine whether the output from the LLM indicates the identification of at least one candidate POI category label may occur prior to the extraction of at least one candidate POI category label at S235, and / or may result from extraction of at least one candidate POI category label at S235 being deemed impossible (e.g., if the LLM output consists of meaningless data). Performing such a check in at least one stage of the process avoids useless output from the LLM leading to undesirable results.

[0087] In order to still be able to provide a result when no useful candidate POI category label is considered to be present in the LLM output, that is, in order to consider the possibility of failure in identification of at least one candidate POI category label, using the character string 10 for identifying at least one point of interest or POI, S200 may include searching the POI database 21 and / or the map database 20 based on at least a portion of the character string 10 at S260.

[0088] In some embodiments, if it is determined at S240 that the extracted candidate POI category label does not correspond to an existing POI category label, then it may be considered to query the LLM at least once more using the same input and / or LLM output ( S230 ).

[0089] Although Figure 3This is primarily indicated in the case where the extracted candidate POI category mark does not correspond to an existing POI category (and / or known POI category mark), and / or in the case where the output from the LLM indicates a failure of identification of at least one candidate POI category mark, but it is not excluded that such a search S260 may be performed in more or even all cases, where it will not be unique but may be one of the sources of multiple geospatial locations identified at S300, which are then filtered and / or ranked according to various factors.

[0090] For example, the string 10 may contain the name of a specific POI, i.e. a name identifying a single / unique place (e.g. "Rijksmuseum Amsterdam"), rather than a name applicable to a subset of POIs in a certain POI category 22 (e.g. "Starbucks"). In the former case, the specificity in the string 10 indicates that the user may not be particularly interested in other museums; the POI may be identified (S260) by searching the POI database 21 (and / or the map database 20) based on at least part of the string 10. On the other hand, in the latter case, it may be advantageous to provide not only geospatial locations corresponding to Starbucks franchises, but also geospatial locations corresponding to other places that provide (takeaway) coffee services; therefore, in this case, it may be advantageous to identify the POI by searching the POI database 21 (and / or the map database 20) based on the string 10 and based on the POI category "cafe", where this POI category can be identified, for example, by querying the LLM at S230, or - if the LLM has been previously queried based on the term "Starbucks" - from a log of previous LLM inputs and associated outputs.

[0091] It may be that the string 10 received from the user is not intended to identify a POI category. In particular, the string 10 may include (at least part of) an address. Therefore, the method may include using a heuristic at S105 to identify strings that do not indicate (or are considered unlikely to indicate) a POI category. For addresses, such heuristics may, for example, involve checking whether the string 10 contains numbers, and / or whether the syntax follows a common address syntax.

[0092] As mentioned previously, LLM queries are "expensive" in terms of time, energy, and computational requirements. Therefore, it may be advantageous to combine LLM queries with a more computationally efficient method for identifying at least one POI using the string 10.

[0093] The method may include checking, at S210, whether at least part of the string 10 satisfies a first matching criterion with at least one of a predetermined plurality of POI categories 22. For example, determining the first matching criterion may involve comparing the string 10 with a set of POI category tags 23, preferably including at least one POI category tag 23 per POI category 22. This may involve checking whether there is an exact match between any POI category tag and (part of) the string 10; the matching criterion is then whether there is a correspondence / match between at least part of the string 10 and at least one POI category tag 23. A certain degree of "fuzziness" may be employed to take into account spelling variations / errors etc. If at least one "match" is established, i.e. if it is determined that at least part of the string 10 satisfies the first matching criterion for at least one POI category tag 23, then at least one POI may be identified, and the method may include identifying, at S225, at least one POI whose POI category 22 is associated with at least one matching POI category tag 23.

[0094] The method may include checking whether at least part of the string 10 satisfies a second matching criterion with at least one of the predetermined plurality of POI categories at S220. Preferably, the second matching criterion is a more complex matching criterion, which may be computationally more complex to calculate / evaluate, but which may have a higher accuracy (in the sense of fewer false negatives) and / or be able to find more subtle similarities.

[0095] Determining at S220 whether at least a portion of the string 10 satisfies a second matching criterion with at least one of the predetermined plurality of POI categories 22 may include calculating a similarity parameter between the string 10 and the POI category 22 for each of the predetermined plurality of POI categories 22 (and / or POI category tags 23). Then, the second matching criterion may be whether the similarity criterion exceeds a first threshold for at least one POI category 22. The method may include identifying at least one POI based on at least one POI category 22 for which the similarity criterion exceeds the first threshold at S225. The value of the similarity criterion may be considered in the final filtering / ranking of the results.

[0096] Determining at S220 whether at least a portion of the character string 10 satisfies a second matching criterion with at least one of the predetermined plurality of POI categories 22 may include converting at least a portion of the received character string 10 into a search vector and calculating a distance between the search vector and the vector representation of the predetermined plurality of POI categories 22. Such vector indexing methods for evaluating semantic similarity are known and will not be described in detail here. It should be noted that if such methods are used, the vector indexing may be done based on the POI category 22 (e.g., using a preferred POI category tag); considering all existing POI category tags 23 is unlikely to produce better results because POI category tags 23 associated with the same POI category 22 may be very similar semantically. If a vector index or another semantic similarity measure is used, the matching criterion may be determined based on the number of POI categories 22 whose distance does not exceed a second threshold. If the distance to a certain POI category is very small, then it is very likely that useful results will be found based on this category; however, when using a vector index or other semantic similarity measure, it may also be useful (in addition or alternatively) to consider relative distances (which may indicate that a term may have an ambiguous meaning).

[0097] Optionally, such semantic similarity measures may be used to update the list of POI category tags. For example, where a candidate POI category tag does not satisfy the first matching criterion with any existing POI category tag 23 but is determined to have a strong semantic similarity / small vector distance with an existing POI category in the predetermined plurality of POI categories 22 upon evaluation of the second matching criterion, the candidate POI category tag may be considered for addition as a new POI category tag for this POI category.

[0098] Figure 4 An embodiment is shown in which determining whether at least a portion of the string 10 satisfies a second matching criterion S220 with at least one of the predetermined plurality of POI categories 22 is in response to determining at S210 that the string 10 does not satisfy the first matching criterion with any of the predetermined plurality of POI categories 22, and is therefore only done, for example, if no portion of the string 10 has an unambiguous correspondence with a POI category label 23. This may have the advantage of trying the least computationally efficient (and least complex) approach first, solving a certain proportion of queries, and gradually moving to more complex approaches only when necessary. However, evaluating only the first matching criterion or only the second matching criterion (i.e., omitting S210 or S220); performing both evaluations in parallel, optionally interrupting the more complex evaluation at S220 if a result can be found based on S210; or various other combinations / permutations are also possible.

[0099] Optionally, querying the LLM to find output associated with the identification of at least one POI category tag S230 is in response to determining that the string 10 does not satisfy the first matching criterion at S210 and / or in response to determining that the string 10 does not satisfy the second matching criterion. It may be advantageous to reserve the use of the LLM only for situations where no (faster and cheaper but less creative) method of parsing the query string 10 can be used.

[0100] Additionally and / or alternatively, before querying the LLM for output associated with the identification of at least one POI category tag at S230 , a log of previous LLM queries may be consulted to see whether a similar or identical string has been previously used to query the LLM, and if so, which output was returned.

[0101] However, in particular if it is considered important to expand the POI database 21 , it is not excluded that the LLM may be queried even if a previous analysis of the string 10 has produced at least one POI category 22 having a certain similarity to at least a portion of the string 10 .

[0102] pass Figure 5 Methods leading to the expansion / updating / enrichment of POI information (eg, as part of the POI database 21) are further described.

[0103] Figure 5 Rendering is performed by executing Figures 1 to 4 In the method described in , a number of POIs have been identified based on the search string 10. In particular, in Figure 5 , the geospatial location 50, 50' corresponding to the respective POI has been identified by searching the POI database at S252 based on the candidate tags 60, 60' extracted from the output of the LLM from the at least partial query based on the string 10. This may also include the case where the candidate tags 60, 60' were extracted from the LLM output at an earlier time, and later retrieved from the search log to identify the specific POI / geospatial location 50 / 50'. The candidate tags 60, 60' may be the same tag, or may be different tags. In addition, another method may have been used, such as by searching the POI database 21 and / or the map database 20 based on at least a portion of the string 10 to identify at least one other geospatial location 51.

[0104] At S260, a number of results may be displayed to the user in a selectable manner. To this end, the method may involve selecting a plurality of geospatial locations (e.g., based on the user's current location, or any of a variety of factors) and providing data for displaying the selected plurality of geospatial locations to the user. It should be noted that in the case where at least a portion of the method is performed on a device provided with display and input capabilities, the display may be part of the method; however, the data for display may be provided to a separate device including display and input capabilities in a known manner. Optionally, a corresponding candidate category label may also be displayed to clarify to the user what the specific POI is identified on. Selection of a POI may, for example, result in providing navigation instructions to this POI, and / or providing other information about the selected POI.

[0105] The selection of a geospatial location / POI identified by searching the POI database 21 or map database 20 based on the candidate POI category tags extracted from the LLM output, i.e., geospatial location 50 or 50', may indicate that the extracted candidate POI category tags are indeed related to the string 10, and / or the extracted candidate POI category tags are relevant for the identified POI. This is particularly the case if such geospatial location is selected from a set of geospatial locations that also includes geospatial locations identified without the use of LLM, as this may indicate deficiencies in the existing list of POI database 21 and / or POI categories 22 and / or POI category tags 23.

[0106] Thus, the method may include, in response to receiving a selection of a geospatial location associated with a candidate POI category tag at S265, storing data indicating an association between the selected POI and the corresponding candidate POI category tag in the data storage device 25 at S270. Figure 5 and Figure 6 Although depicted as a separate data storage device, the data storage device 25 may be included in the POI database 21 , the map database 20 , and / or stored together with the POI category 22 and / or the POI category label 23 .

[0107] Figure 6Methods of updating POI information using such a data storage device 25, such as the POI database 21 (which directly or indirectly also requires updating a predetermined plurality of POI categories 22) are described. Such methods may be performed once the methods as described above have been performed a sufficient number of times to populate the data storage device 25 to the extent that the data therein may be analyzed. For example, at least a portion of the data storage device 25 may be analyzed based on the number of entries marked for a certain candidate POI category exceeding a predetermined threshold; alternatively or in addition, analysis may occur once the data storage device 25 as a whole reaches a certain size / number of entries and / or may occur periodically once a predetermined period of time has passed.

[0108] The method of updating POI information may include grouping POIs associated with a candidate POI category tag in the data storage device 25 at S410. This may include grouping POIs associated with exactly the same candidate POI category tag, but as described elsewhere, a certain degree of fuzziness may also be employed, such as to take into account different spellings ("harbor" / "harbour").

[0109] The method may then include, at S420 , retrieving corresponding POI categories for all POIs in a certain group from the POI database 21 ; and at S430 , determining the frequencies of occurrence of these categories.

[0110] It should be noted that other analyses may occur before, during, or after these steps. For example, the data store 25 may be deduplicated to remove instances where the same candidate POI category label results in the same POI / geospatial location - but on the other hand, duplicate selections may also be an indication of usefulness, and it may be useful to weigh such duplicate selections more heavily in the analysis.

[0111] The method may include determining at S431 whether a first criterion is satisfied. The first criterion may be, for example, that a large proportion (e.g., a majority, or more than a certain threshold) of the POIs associated with a certain candidate POI category mark 60, 60' in the data storage device 25 are associated with a certain POI category. In this case, the candidate POI category mark is likely to be frequently used by users who want to find POIs of this category. Therefore, the method may include, in response to satisfying the first criterion, adding the candidate POI category mark to the list of POI category marks 23 as a mark associated with this POI category.

[0112] The method may include determining at S432 whether a second criterion is satisfied. The second criterion may be selected so that it is satisfied in the case where there is no explicit POI category corresponding to this tag but a new category may be needed. The second criterion may optionally be that the first criterion is not satisfied.

[0113] More generally: Once enough data has been collected, statistical analysis can be used to identify potential new POI category labels and potential new POI categories. In particular, let us assume that the data storage device contains n instances of a certain candidate POI category label, which are linked to m POIs selected by the user (where m and n are integers, and m≤n). In the existing POI database, each of the m POIs may be associated with at least one existing POI category c. Each of these POI categories may appear in c i times. The frequency of occurrence can be indicated as c i / m. It is also possible to calculate this frequency in a different way, for example by counting the category each time the POI appears, and relating the total number of category occurrences to n.

[0114] If a large number of the m selected POIs are associated with the same category, for example, if c i / m exceeds a certain threshold (e.g., 0.6, 0.7, 0.8, 0.9) and / or if it is significantly higher than the c of other categories associated with the selected result found based on this candidate POI category label j / m, then the candidate POI category label that leads to these results may be the category c i The "first criterion" is preferably a criterion indicating that a certain candidate POI category label appears to produce a successful result that may be associated with an existing POI category.

[0115] It is also possible that the frequency of occurrence does not indicate a clear one-to-one correspondence. Successful results may correspond to several different categories. It is also possible that the most common category contains two or more categories, for example because the term has multiple meanings (see the Dutch "hair salon", which means "barber shop", but is also the name of a popular fast food) or because the term has a broad meaning, covering several more specific subcategories (such as "art", which may be related to "museum", "art gallery" and "art school"). The second criterion is preferably selected to identify such cases.

[0116] More complex statistical analysis may also be performed, for example based on displayed but unselected POIs.

[0117] In another example, in the case where at least some POI categories are structured in a hierarchical manner, i.e., some POI categories are subcategories of other categories, the second criterion may be based (at least in part) on a comparison of the frequency of a certain POI category with the frequency of subcategories of this POI category. For example, if we assume that the POI category "public transport stops" contains the subcategories "bus stops" and "tram stops", then the string may cause the LLM to output "water taxi stops", and identify and select POIs that have "public transport stops" as a category but are not associated with any of the existing subcategories of this POI category.

[0118] In response to satisfying the second criterion, the method may include, at S442, creating a new category in the POI database and associating the new POI category with the POI having an association with the candidate POI category tag stored in the data storage device 25. For example, in the above example, the result may be the creation of a new POI category "water taxi station" as a subcategory of the existing POI category "public transport station". If the POI categories 22 are stored as part of the POI database 21, then this automatically results in an update to the plurality of predetermined POI categories; if not, then the method may include adding the new category to the predetermined plurality of POI categories 22 (or causing it to be added), with the candidate POI category tag as the associated POI category tag.

[0119] It should be understood that the described methods have been shown as individual steps performed in a particular order. However, those skilled in the art will appreciate that these steps can be combined or performed in a different order while still achieving the desired results.

[0120] Figure 7 Various components of the system 1000 for implementing the methods described above are illustrated. It should be noted that the various components may be combined / juxtaposed in various ways.

[0121] It should be understood that embodiments of the present invention can be implemented using various information processing systems. In particular, although the accompanying drawings and their discussion provide exemplary computing systems and methods, these are only presented to provide useful references when discussing various aspects of the present invention. Embodiments of the present invention can be implemented on any suitable data processing device, such as a personal computer, a laptop computer, a personal digital assistant, a mobile phone, a set-top box, a television, a server computer, etc., and include combinations of more than one such device of the same type that are not necessarily implemented. Of course, for the purpose of discussion, the description of the system and method has been simplified, and it is only one of the many different types of systems and methods that can be used for embodiments of the present invention. It should be understood that the boundaries between the logic blocks are only illustrative, and alternative embodiments can merge logic blocks or elements, or can apply alternative functional decompositions to various logic blocks or elements.

[0122] It should be understood that the functionality mentioned above can be implemented as one or more corresponding modules as hardware and / or software. For example, the functionality mentioned above can be implemented as one or more software components for execution by a processor of a system. Alternatively, the functionality mentioned above can be implemented as hardware, such as on one or more field programmable gate arrays (FPGAs) and / or one or more application specific integrated circuits (ASICs) and / or one or more digital signal processors (DSPs) and / or other hardware arrangements. The method steps implemented in the flowcharts contained herein or as described above can each be implemented by a corresponding respective module; multiple method steps implemented in the flowcharts contained herein or as described above can be implemented together by a single module.

[0123] It should be understood that as long as the embodiments of the present invention are implemented by a computer program, the storage media and transmission media carrying the computer program form aspects of the present invention. A computer program may have one or more program instructions or program codes, and when executed by a computer, the program instructions or program codes implement embodiments of the present invention. The term "program" as used herein may be a sequence of instructions designed to be executed on a computer system, and may include subroutines, functions, procedures, modules, object methods, object implementation schemes, executable applications, applets, service programs, source codes, object codes, shared libraries, dynamic link libraries, and / or other sequences of instructions designed to be executed on a computer system. The storage medium may be a disk (such as a hard disk or a floppy disk), an optical disk (such as a CD-ROM, a DVD-ROM, or a Blu-ray disk), or a memory (such as a ROM, RAM, an EEPROM, an EPROM, a flash memory, or a portable / removable memory device), etc. The transmission medium may be a communication signal, a data broadcast, a communication link between two or more computers, etc.

[0124] Now return to Figure 7 In the embodiment illustrated in , the query module 100 is configured to analyze the string 10 to identify at least one geospatial location. Although it is depicted as a single module, it is not excluded that its functionality is distributed over more than one computing device. The string 10 is received at an input / output module 200 (e.g., a navigation system or a smartphone), which includes an input module 210 and a display module 220. The input module 210 is configured to receive the string and optionally receive a selection of a geospatial location from a user, and the display module 220 is configured to display at least one identified geospatial location. Both the input module 210 and the display module 220 may be embodied as a touch screen display.

[0125] The query module 100 and the input / output module 200 may be provided by the same device (e.g. a navigation system or a smartphone); if not, it is provided with communication capabilities (illustrated by the double arrows) so that the received string 10 can be transmitted to the query module 100 and the at least one identified geospatial location (possibly including data required for displaying this geospatial location) can be transmitted to the input / output module 200.

[0126] To be able to identify at least one geospatial location, the query module 100 is configured to be able to communicate with the LLM module 300 on which the Large Language Model LLM is implemented. The query module 100 and the LLM module 300 may be implemented on the same device; if not, communication capabilities are provided so that the query module 100 can transmit queries / prompts to the LLM module 300 and can receive LLM output in exchange.

[0127] The system 1000 may further include at least one data storage module 400, 110. Figure 7 In FIG. 4 , the data storage module 400 is depicted as an external data storage module implemented on a separate device, while the data storage module 110 is depicted as an internal data storage module as part of the query module 100 , but this should not be considered limiting.

[0128] The external data storage module 400 may, for example, store the map database 20 and / or the POI database 21 , which may be accessed by the query module 100 , for example, to retrieve POIs associated with a certain category.

[0129] The internal data storage module 110 may, for example, store a list of predetermined POI categories 22 and / or POI category labels 23. The POI categories 22 may be derived from the POI database 21 and may also be part of the POI database 21. A set of POI category labels 23 may be maintained by the query module 100. The internal data storage module 110 may store a data storage device 25 storing data indicating an association between a selected POI and a corresponding candidate POI category label - however, this may also be stored on the external data storage module 400, for example as part of the POI database 21. The internal data storage module 110 may store a log of previous LLM queries and responses (not depicted).

[0130] The system 1000 may further include an analysis module 120. Figure 7In this case, this is depicted as part of the query module 100, but it can also be implemented separately; in either case, it can be communicatively coupled in some way to the data storage module 400 (or 110) that stores the POI database 21; to the data storage module 110 (or 400) that stores the POI categories 22 and the POI category markers 23; and to the data storage module 110 (or 400) that stores the data storage device 25.

[0131] Figure 8 Show an example screen that can be displayed in the context of the method described above. The display itself can form part of the method; the method can additionally or alternatively include providing data for displaying such a screen.

[0132] It should be noted that in this figure as well as Fig.9A and 9B In both cases, the boxes outlined with a dashed border are not part of the content to be displayed, but are used to indicate sub-sections of the content to be displayed.

[0133] Specifically, Figure 8 Show the possible results of a search string corresponding to the string 10 corresponding to "Food". In this example, the search bar 81 and the search results 82 are displayed in combination with the map 80, and in an embodiment, the map 80 may include markers for some or all of the displayed results. In this example, for each of the results 82, an icon 84 is displayed, which indicates the type of the result. For example, it can be an icon based on the corresponding POI category.

[0134] In the displayed example, some POIs are directly identified based on the string 10. For example, the result 85 contains the word "Food" as part of the POI name. Other results (such as the POI 86) do not contain the word "Food" in their names, but can be found in other ways, such as based on the POI category associated with the POI category marker "Food". In addition to the results corresponding to the POIs, the result list may also contain the names of POI categories, such as "International Restaurants" 83; selecting this POI category then changes the result list according to the selected category (whether by completely replacing it; replacing some results; or supplementing the result list).

[0135] However, not all strings 10 are so easy to parse. Fig.9A Show a possible result list for the string Picasso 91. In this example, assume that we are in a situation where "Picasso" does not correspond to an existing POI category marker. Therefore, the list of results will mainly contain results that contain the word "Picasso" in the POI name and / or the street name, such as the Picasso Aquarium 95.

[0136] Fig. 9BShows how to use LLM query to identify potentially more relevant results. Specifically, in this scenario, assume that a query to LLM returns two candidate POI category tags, namely "art gallery" 93 and "car" 94. It should be noted that this may cause the correspondence between "Picasso" and "art gallery" and "Picasso" and "car" to be stored in the query log, so that the query to LLM 40 may not be needed next time. In this scenario, neither of the two candidate POI category tags corresponds to an existing POI category tag. Fig.9A , the user has selected "art gallery" (as indicated by the color difference), resulting in the results including art gallery 96, as well as other results for "art gallery" that can be found by querying the map database and / or the POI database. If the user selects result 96, then the association between "art gallery" and this particular POI may be stored in the data storage device 25. Result 96 may already have an associated POI category, namely the POI category "museum". If other queries result in the candidate POI category tag "art gallery", and the user subsequently selects POI 96 or other POIs associated with the POI category "museum", this may ultimately result in "art gallery" being added as a new POI category and / or POI category tag, and / or the category associated with the relevant selected POI being updated / changed.

[0137] Although not depicted, advantageous results may also be achieved for strings 10 in languages ​​not (yet) represented in existing POI category tags, as the LLM may be able to find the most appropriate existing POI category. Furthermore, the proposed method may assist in processing more complex entries. For example, querying the LLM to find candidate POI category tags for the string "I need a car for the week" may result in the identification of "Car rental facilities", resulting in the identification of POIs that have not been found based on a simple search of the map database and / or POI database. Similarly, querying the LLM to find candidate POI category tags for the string "Rent a stand-up paddle board" may result in the identification of the POI category "Water Sports".

[0138] This description and the drawings illustrating aspects and embodiments of the present invention should not be considered as limiting the claims defining the protected invention. In other words, although the present invention has been described and described in detail in the drawings and the foregoing description, such description and description should be considered illustrative and non-restrictive. Various mechanical, compositional, structural, electrical and operational changes may be made without departing from the spirit and scope of the present description and claims. In some examples, well-known circuits, structures and techniques are not shown in detail in order not to obscure the present invention. Therefore, it should be understood that those of ordinary skill in the art may make changes and modifications within the scope and spirit of the following claims. In particular, the present invention encompasses other embodiments having any combination of features from the different embodiments described above and below.

[0139] The present disclosure also encompasses all other features individually shown in the drawings, even though they may not be described in the preceding or following description. Furthermore, individual alternatives to the embodiments depicted in the drawings and descriptions of their features and individual alternatives may be disclaimed from the subject matter of the present invention or the disclosed subject matter. The present disclosure includes subject matter consisting of the features defined in the claims or embodiments as well as subject matter comprising said features.

[0140] The term "comprising" does not exclude other elements or process blocks, and the indefinite article "a" or "an" does not exclude a plurality. A single unit or process block may fulfill the functions of several features stated in the claims. The fact that certain measures are stated in mutually different dependent claims does not indicate that a combination of these measures cannot be utilized. Components described as coupled or connected may be directly electrically or mechanically coupled, or they may be indirectly coupled via one or more intermediate components. Any reference signs in the claims should not be interpreted as limiting the scope.

[0141] The foregoing detailed description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the technology to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The described embodiments were chosen in order to best explain the principles of the technology and its practical application, thereby enabling others skilled in the art to best use the technology in various embodiments and with various modifications suitable for the particular use contemplated. The scope is intended to be defined by the claims appended hereto.

Claims

1. A computer-implemented method for identifying at least one geospatial location based on user input, comprising: receiving a character string from a user as the user input; Using the character string to identify at least one point of interest (POI); and identifying a geospatial location corresponding to the at least one identified POI as the at least one geospatial location; The use of the string includes: A large language model (LLM) is queried based on at least a portion of the character string and a plurality of predetermined POI categories to find an output associated with an identifier of at least one POI category tag.

2. The method of claim 1, wherein using the string further comprises extracting at least one candidate POI category label from the output of the LLM; The at least one POI is identified based on the at least one candidate POI category label.

3. The method according to claim 2, wherein identifying the at least one POI based on the at least one candidate POI category tag comprises: determining whether the at least one candidate POI category label corresponds to at least one POI category of the predetermined plurality of POI categories, the method optionally further comprising, in response to determining that the at least one candidate POI category label corresponds to at least one of the predetermined plurality of POI categories, identifying the at least one POI based on the at least one corresponding POI category of the predetermined plurality of POI categories.

4. The method of claim 3 , further comprising, in response to determining that the at least one candidate POI category label does not correspond to at least one POI category of the predetermined plurality of POI categories, identifying the at least one POI by searching a POI database and / or a map database based on the candidate POI category label.

5. The method of claim 4, further comprising providing data for displaying the at least one identified POI to a user in a selectable manner, optionally combined and / or grouped together with the corresponding candidate POI category label.

6. The method of claim 5 , further comprising, in response to receiving a selection of one of the at least one POI identified by searching a POI database and / or a map database based on the candidate POI category label, storing data indicating an association between the selected POI and the corresponding candidate POI category label in a data storage device.

7. The method of any one of claims 3 to 6, further comprising, in response to determining that the at least one candidate POI category marker does not correspond to at least one POI category of the predetermined plurality of POI categories, providing data for displaying the at least one candidate POI category marker to a user in a selectable manner, the method optionally comprising, in response to receiving a selection of at least one displayed candidate POI category marker, providing at least one POI corresponding to the selected POI category marker.

8. The method according to any one of claims 1 to 6, wherein using the character string for identifying at least one POI further comprises: determining whether at least a portion of the character string satisfies a matching criterion with at least one of the predetermined plurality of POI categories, wherein optionally each POI category is associated with at least one POI category tag; Optionally, querying the LLM to find an output associated with an identification of at least one POI category tag is in response to determining that the string does not satisfy the matching criteria. 9 . The method of claim 8 , further comprising, in response to determining that at least a portion of the string satisfies the matching criteria for at least one POI category, identifying the at least one POI based on the at least one POI category.

10. The method of claim 8, wherein determining whether at least a portion of the character string satisfies a matching criterion with at least one of the predetermined plurality of POI categories comprises at least one of the following: For each POI category tag, identifying whether there is a corresponding relationship between the POI category tag and at least part of the string, wherein optionally the matching criterion is whether there is a corresponding relationship between at least part of the string and at least one POI category tag; for each POI category of the predetermined plurality of POI categories, calculating a similarity parameter between the character string and the POI category, wherein optionally the matching criterion is whether the similarity parameter exceeds a first threshold value for at least one POI category; and At least part of the received string is converted into a search vector, and a distance between the search vector and vector representations of the predetermined plurality of POI categories is calculated, wherein optionally the matching criterion is determined based on the number of POI categories for which the distance does not exceed a second threshold.

11. The method of any one of claims 1 to 6, wherein using the string further comprises, in response to the output from the LLM indicating a failure of the identification of at least one candidate POI category label, identifying the at least one POI by searching a POI database and / or a map database based on at least a portion of the string.

12. The method of any one of claims 1 to 6, further comprising providing data for displaying the at least one geospatial location to a user, and wherein: Providing data for displaying the at least one geospatial location to a user includes providing data for visually indicating, for at least one geospatial location, that the geospatial location is identified based on a POI category, optionally including visually indicating the POI category based on which the geospatial location is identified; When a plurality of geospatial locations are identified, providing data for displaying the plurality of geospatial locations to a user includes providing data for grouping and / or ranking the plurality of identified geospatial locations according to the corresponding POI categories; The method further comprises providing data for selectably displaying a visual indication of at least one POI category identified using the string, and in response to receiving a selection of a POI category, identifying at least one geospatial location based on the selected POI category and providing data for displaying the at least one geospatial location to a user, and / or When a plurality of geospatial locations are identified, providing data for displaying the plurality of geospatial locations to a user comprises providing data for selectably displaying a visual indication of at least one POI category corresponding to at least one of the plurality of identified geospatial locations, wherein optionally the method further comprises, in response to receiving a selection of a POI category, filtering and / or re-ranking the plurality of geospatial locations based on the selected POI category.

13. A method according to any one of claims 1 to 6, comprising providing data for displaying at least one of the identified geospatial locations to a user in a selectable manner, and in response to receiving the selection of at least one geospatial location, providing data indicative of instructions for navigating to the selected geospatial location.

14. A method for updating point of interest (POI) information using a POI database, wherein a plurality of POIs are associated with at least one of a plurality of predetermined POI categories, wherein the POI information includes at least one POI category tag associated with each POI category, the method comprising: accessing a data storage device obtainable by the method according to claim 6; For a given candidate POI category label: determining a frequency of occurrence in the POI database of at least one of the predetermined plurality of POI categories associated with a POI having an association with the candidate POI category label stored in the data storage device; In response to determining that the occurrence frequency satisfies a first criterion, adding the candidate POI category tag as a POI category tag to the at least one of the predetermined plurality of POI categories; and In response to determining that the frequency of occurrence satisfies a second criterion, the candidate POI category label is added to the database as a new POI category and the new POI category is associated with the POI having an association with the candidate POI category label stored in the data storage device.

15. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method according to any one of the preceding claims.

Images