Provide search results based on combined queries

Abstract

The technology for providing search results may include determining a first entity type, a second entity type, and a relationship type based on a combined query. The technology may also include identifying a node of a knowledge graph corresponding to an entity reference of the first entity type and an entity reference of the second entity type. The technology may also include determining an attribute value corresponding to the relationship type for each entity reference of the first entity type and each entity reference of the second entity type from the knowledge graph. The technology may also include comparing the attribute value of each entity reference of the first entity type with the attribute value of each entity reference of the second entity type. The technology may also include determining one or more result entity references from the entity reference of the first entity type based on the comparison.

Description

[0001] This application is a divisional application of the PCT invention patent application with the application date of December 12, 2012, application number 201280078167.9, and invention name "Providing search results based on combined query". Background Art

[0002] The present disclosure relates to providing search results based on combined queries. A search engine receives a search query from a user and provides search results using, for example, a list of text links. Search engines typically solve queries such as [Starbucks near San Francisco Airport] or [Movies shot during World War II] by returning a collection of results based on a single, fixed location criterion or based on a single, fixed time criterion. Summary of the invention

[0003] In some implementations, a computer-implemented method includes: using at least one processor to determine a first entity type, a second entity type, and a relationship based on a combined query. The computer-implemented method includes: using at least one processor to identify a node of a knowledge graph corresponding to an entity reference of the first entity type and an entity reference of the second entity type. The computer-implemented method includes: using at least one processor to determine an attribute value corresponding to the relationship for each entity reference of the first entity type and each entity reference of the second entity type from the knowledge graph. The computer-implemented method includes: using at least one processor to compare the attribute value of each entity reference of the first entity type with the attribute value of each entity reference of the second entity type. The computer-implemented method includes: using at least one processor to determine one or more result entity references from the entity reference of the first entity type based on the comparison. Other implementations of this aspect include corresponding systems configured to perform the actions of the method.

[0004] In some implementations, a computer-implemented method includes: receiving user input indicating a first entity type, a second entity type, and a relationship between a plurality of entity references of the first entity type and a plurality of entity references of the second entity type that define a criterion. The computer-implemented method includes: identifying, from a knowledge graph, a plurality of pairs of entity references of the first entity type and the second entity type that satisfy the criterion. The computer-implemented method includes causing presentation of a representation of an entity reference to at least one of the entity references of the first entity type and the entity references of the second entity type from the plurality of pairs. Other implementations of this aspect include corresponding systems configured to perform the actions of the method.

[0005] In some implementations, a method implemented by one or more processors includes:

[0006] Determine based on combined query:

[0007] The first entity type,

[0008] The second entity type, and

[0009] distance relationship;

[0010] Identify from the data structure:

[0011] a plurality of first candidate entity references of the first entity type, and

[0012] a second entity reference of the second entity type;

[0013] determining, from the data structure, an attribute value corresponding to a distance relationship between each of the plurality of first candidate entity references and the second entity reference;

[0014] comparing the attribute value of each of the first candidate entity references with the attribute value of the second entity reference;

[0015] determining one or more resultant first entity references from the first candidate entity references based on the comparing, and

[0016] In response to receiving the combined query and displaying it as a mark on a map of a graphical user interface:

[0017] the second entity references, and

[0018] At least one of the one or more result first entity references.

[0019] In some implementations, a method implemented by one or more processors includes:

[0020] Determine that the received combined query includes:

[0021] The first entity type,

[0022] The second entity reference, and

[0023] distance relationship;

[0024] Identify from the data structure:

[0025] a plurality of first candidate entity references of the first entity type, and

[0026] each corresponding position of the first candidate entity reference and the second entity reference;

[0027] comparing the corresponding position of each of the first candidate entity references with the corresponding position of the second entity reference to determine a corresponding distance between each of the first entity references and the second entity reference;

[0028] determining the one or more resulting first entity references from the first candidate entity references based on corresponding distances of one or more resulting first entity references that satisfy a distance criterion determined based on the distance relationship, and

[0029] In response to the combined query, the following is displayed as a label on a map of a graphical user interface:

[0030] the second entity references, and

[0031] At least one of the one or more result first entity references.

[0032] In some implementations, a method implemented by one or more processors includes:

[0033] receiving user input indicating a first entity type, at least one second entity reference, and a distance relationship between a plurality of first entity references of the first entity type and the at least one second entity reference, wherein the relationship defines a distance criterion;

[0034] determining from a data structure a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion; and

[0035] In response to determining a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion:

[0036] To display it as a callout on the GUI map:

[0037] the second entity references, and

[0038] A plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion.

[0039] In some implementations, a system for providing search results includes:

[0040] Databases; and

[0041] One or more processors configured to perform operations comprising:

[0042] Determine based on combined query:

[0043] The first entity type,

[0044] The second entity type, and

[0045] distance relationship;

[0046] Identify from the database:

[0047] a plurality of first candidate entity references of the first entity type, and

[0048] a second entity reference of the second entity type;

[0049] determining, from the database, an attribute value corresponding to a distance relationship between each of the plurality of first candidate entity references and the second entity reference;

[0050] comparing the attribute value of each of the first candidate entity references with the attribute value of the second entity reference;

[0051] determining one or more resultant first entity references from the first candidate entity references based on the comparing, and

[0052] In response to receiving the combined query and displaying it as a label on a map of a graphical user interface:

[0053] the second entity references, and

[0054] At least one of the one or more result first entity references.

[0055] In some implementations, a system for providing search results includes:

[0056] Databases; and

[0057] One or more processors configured to perform operations comprising:

[0058] Determine that the received combined query includes:

[0059] The first entity type,

[0060] The second entity reference, and

[0061] distance relationship;

[0062] Identify from the database:

[0063] a plurality of first candidate entity references of the first entity type, and

[0064] Each of the first candidate entity references and the second entity reference corresponds to

[0065] Location;

[0066] comparing the corresponding position of each of the first candidate entity references with the corresponding position of the second entity reference to determine a corresponding distance between each of the first entity references and the second entity reference;

[0067] determining the one or more resulting first entity references from the first candidate entity references based on corresponding distances of one or more resulting first entity references that satisfy a distance criterion determined based on the distance relationship, and

[0068] To display it as a callout on the GUI map:

[0069] the second entity references, and

[0070] At least one of the one or more result first entity references.

[0071] In some implementations, a system for providing search results includes:

[0072] Databases; and

[0073] One or more processors configured to perform operations comprising:

[0074] receiving user input indicating a first entity type, at least one second entity reference, and a distance relationship between a plurality of first entity references of the first entity type and the at least one second entity reference, wherein the relationship defines a distance criterion;

[0075] determining from the database a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion; and

[0076] In response to determining a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion:

[0077] To display it as a callout on the GUI map:

[0078] the second entity references, and

[0079] A plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The above and other features of the present disclosure, its nature, and various advantages will be apparent upon consideration of the following detailed description taken in conjunction with the accompanying drawings, in which:

[0081] Figure 1 is a block diagram of an illustrative search system for solving combined queries according to some implementations of the present disclosure;

[0082] Figure 2Two illustrative data structure representations that may be helpful in solving combined queries according to some implementations of the present disclosure are shown;

[0083] Figure 3 shows an illustrative knowledge graph including nodes and edges according to some implementations of the present disclosure;

[0084] Figure 4 shows an illustrative knowledge graph portion according to some implementations of the present disclosure;

[0085] Figure 5 Another illustrative knowledge graph portion according to some implementations of the present disclosure is shown;

[0086] Figure 6 is an illustrative first screenshot of a user interface of a search system according to some implementations of the present disclosure;

[0087] Figure 7 is an illustrative second screenshot of a user interface of a search system according to some implementations of the present disclosure;

[0088] Figure 8 is an illustrative third screenshot of a user interface of a search system according to some implementations of the present disclosure;

[0089] Fig. 9 is yet another illustrative screenshot of a user interface of a search system according to some implementations of the present disclosure;

[0090] Fig.10 is an illustrative screenshot of a user interface of a search system showing a timeline according to some implementations of the present disclosure;

[0091] Fig.11 is a flow chart of illustrative steps for identifying result entity references based on a combined query according to some implementations of the present disclosure;

[0092] Fig.12 is a flow chart of illustrative steps for storing data in a data structure for use with a combined query according to some implementations of the present disclosure;

[0093] Fig.13 is a flow chart of illustrative steps for displaying at least some result entity references based on a combined query according to some implementations of the present disclosure;

[0094] Fig.14 An illustrative computer system is shown according to some implementations of the present disclosure; and

[0095] Fig.15 According to some implementations of the present disclosure Fig.14 A block diagram of an illustrative computer system user device. DETAILED DESCRIPTION

[0096] A combined query is a query that includes at least two types of entity references related by relative relationships. In some implementations, a combined query requires identification of at least two types of entity references. As used herein, an entity is a thing or concept that is separate, unique, well-defined and distinguishable. For example, an entity can be a person, a place, an item, an idea, a theme, an abstract concept, a concrete element, other appropriate things, or any combination thereof. In some implementations, search results include results that identify entity references. As used herein, an entity reference is an identifier such as a text or other information that references an entity. For example, an entity can be the physical embodiment of George Washington, and an entity reference is an abstract concept for citing George Washington. Where appropriate, based on context, it will be understood that the term entity as used herein can correspond to an entity reference, and the term entity reference as used herein can correspond to an entity. In some implementations, a search system can identify an entity type associated with an entity reference. An entity type can be a classification or a classification for identifying an entity reference in a data structure. For example, an entity reference "George Washington" can be associated with entity types "American President", "person", and "officer".

[0097] The query [US banks near Japanese restaurants] includes references to two types of places and does not specify any single referenced location, but rather a relative spatial relationship. The query simply indicates that the user wants to get "US banks" results near "Japanese restaurants". However, the query does not specify which "Japanese restaurant" is preferred, nor does it specify which "US banks" are preferred. The likely intent of the query is that the user wants to eat at a Japanese restaurant and stop at an US bank before or after the meal. The user wants search results that include one or more Japanese restaurants that meet the "close to" criteria and one or more best candidates for one or more US banks. Below in Figure 3 , 4 Further discussion of entity references and entity types will be included in the context of ,5.

[0098] Yet another example of a combined query is [companies that went bankrupt during the economic crisis], where the relationship is based on time rather than geographic location. Thus, the search system can identify a first entity type, such as economic crisis, a second entity type, such as corporate bankruptcy filings, and relationships between entity types such as, for example, relative spatial distances or relative time differences. For example, the search system can determine that the entity type is a place, and the relationship is the distance between places. In yet another example, the search system can determine that the entity type is an event, and the relationship is the time between events.

[0099] In addition, the search system can filter a collection of entity references of one or both types based on the query. Search results can be determined based on a comparison of the attributes of the entity references of one or more related entity types. For example, a position value or a time value can be used to compare two entity references to determine whether the necessary criteria are met. The attributes can be stored in any appropriate data structure in a manner that associates the attributes with their corresponding entity references. In some implementations, the search results include entity references in a data structure, a list of documents, a list of document identifiers, a collection of links, images, text, or other content, any other appropriate results, or any combination thereof.

[0100] In some implementations, the pre-generated table can be generated and stored offline and accessed to help identify the result entity references for the combined query. Offline generation of the table refers to generating the table before responding to any specific query. For example, the table may include a data structure corresponding to an N×M array, where the N×M array has N rows and M columns. The row corresponds to a specific type of entity reference such as, for example, a restaurant. Each column corresponds to a specific entity type such as, for example, a bank, an airport, and a cafe. For each row of the array related to the entity reference and each column related to the entity type, the corresponding entry includes, for example, a target entity reference determined to be the entity type closest to the entity reference based on a comparison of attribute values. The target entity reference is an entity reference of a specific entity type determined to be the most consistent with the standard. The entry may also include the actual distance or time between the entity reference corresponding to the row and the target entity reference in the case where the entity is an event. The data structure is generated by comparing each entity reference of each corresponding type with the entity reference corresponding to each row and selecting the target entity reference of the corresponding type closest to the entity reference corresponding to the row. The result entity reference can be presented to the user via a user interface. The user interface may include a display such as, for example, a map or a timeline labeled with the result entity reference. The user interface may allow the user to filter the result entity references. For example, the user may be provided with an option to specify a maximum distance between a first type of entity reference and a second type of entity reference for display.

[0101] It should be understood that in some implementations the search system receives a combined query and identifies result entity references related to more than two entity types, e.g., three, four, five. In the example of a combined query related to three types, the query [Bank of America near Japanese restaurant near ice cream shop] includes references to three types of places.

[0102] Figure 1 is a block diagram of an illustrative search system for solving combined queries according to some implementations of the present disclosure. The search system may include Figure 1. In some implementations, the processing element 100 receives a query 110 from a user input, such as entering a text query, selecting a filter setting, or both. In some implementations, the query 110 is received from a user interface 170. The processing element 100 may include one or more processors having a processor. The processing element 100 may access a data structure 120, a table 130, content L40, any other appropriate information not shown, or any combination thereof. The data structure 120 includes data related to multiple entity references such as, for example, entity names, attributes, entity types, and paths between entities. Table 130 is a data structure that may include an array indexed by entities and entity types. For example, as illustrated, table 130 is indexed by entity references id1 and id2 and entity types T1, T2, T3. An entity includes a target entity reference having a type associated with a column that is closest in distance or time to an entity reference associated with a row, for example. In some implementations, table 130 is pre-generated and stored for subsequent access by processing element 100 to facilitate solving a combined query such as, for example, query 110. Processing element 100 may identify a first entity type, a second entity type, and a type of relationship between entity types. Processing element 100 may determine one or more result entity references 150 of the first type, the second type, or both based at least in part on query 110, data structure 120, table 130, web page 140, any other suitable information, or any combination thereof.

[0103] The data structure block 120 includes a data structure containing information defined at least in part by relationships between them. In some implementations, the data structure block 120 includes any appropriate data structure, data graph, database, index, list, linked list, table, any other appropriate information, or any combination thereof. In an example, the data structure block 120 includes a collection of data stored as nodes and edges in a graph structure. In some implementations, the data structure block 120 includes a knowledge graph. In some implementations, the knowledge graph includes data organized into a graph containing nodes and edges. The data of the knowledge graph may include statements about relationships between entity references, and those statements may be represented as nodes and edges of a graph. Each of the nodes of the knowledge graph contains one or more pieces of data and the edges represent relationships between the data contained in the nodes connected by the edges. Below in Figure 3-5 A specific implementation of the knowledge graph is described in .

[0104] Content block 140 includes stored information from, for example, the Internet. In some implementations, web page block 140 includes web pages, hyperlinks, text, images, audio, video, and other appropriate content on the Internet. In some implementations, content block 140 includes indexed or organized data retrieved from the Internet. For example, web page organized data includes rankings of web pages based on the number of hyperlinks linked to and from the web pages.

[0105] Figure 2 Illustrative data structure representations 200 and 250 are shown that can help solve combined queries according to some implementations of the present disclosure. In some implementations, Figure 1 Table 130 includes one or more data sets such as data structure representations 200 and 250. Data structure representations 200 and 250 can be generated and stored for subsequent reference in response to a combined query. By using references such as data structure representations 200 or 250, the search system can reduce the size of candidate entity reference data and improve the processing time for responding to a combined query.

[0106] The entities of data structure representation 200 include the mile distances between M specific entity references with identifiers ID1, ID2, etc. and the nearest entities of a specific type of N specific types. For example, the distance between entity reference ID M and the nearest entity of type 3 is 10 miles. For example, when the entity of the combined query is explicitly stated or otherwise determinable, data structure representation 200 can be used. The search system can filter possible result entity references based on data structure representation 200, and then determine one or more result entity references. For example, based on the query, the "nearby" relationship can be defined as within 30 miles. For entity reference ID 2, there is no entity reference of type 2 that is "nearby". Therefore, because type 2 is 40 miles away from the nearest entity of entity reference ID 2, the search system can discard entity reference ID 2 in the context of determining the result entity reference. However, for entity reference ID 1, the search system can find entities of type 2 that meet the "close" standard, and therefore the search system can determine the result entity reference only for entity reference ID 1.

[0107] The entities of data structure representation 250 include the distances between M specific entity references with identifiers ID1, ID2, etc. and the nearest entities of a specific type of N specific types and the entity identifiers of the nearest entities of each type. For example, for a given entity reference ID 1, the search system can find the nearest entity of type 2 (which is ID_b1) and the distance between entity references ID 1 and ID_b1 (which is 20 miles). In an illustrative example, let entity reference ID1 be a specific Starbucks restaurant and let type 2 be "airport". The search system can determine whether there is an airport near the Starbucks restaurant and also determine which airport it is. For a query of type [Starbucks near an airport], the restaurant is restricted to Starbucks, but there are no restrictions on airports. References such as data structure representation 250 are particularly useful for responding to combined queries with restrictions of only one entity reference. In some implementations, data structure representation 250 can be generated by modifying data structure representation 200 to include entity identifiers in the entities.

[0108] The entity references, entity types, and relationship information described above that can be used to generate tables or provide, for example, search results can be included as data in the data structure. Figure 3-5 An illustrative knowledge graph is described that may be used by some implementations of the present disclosure. It should be understood that a knowledge graph is merely an example of a data structure that may be used by a search system and any suitable data structure may be used.

[0109] In some implementations, data can be organized in a database using any one or more data structure techniques. For example, data can be organized in a graph containing nodes connected by edges. In some implementations, data can include statements related to the relationship between things and concepts, and those statements can be represented as nodes and edges of a graph. Each node contains one or more pieces of data and the edge represents the relationship between the data contained in the nodes connected by the edge. In some implementations, the graph includes one or more pairs of nodes connected by edges. The edge and thus the graph can be directed, undirected, or both. For example, in the same graph, the edge can be unidirectional, bidirectional, or one or more edges can be undirected and one or more edges can be directed. The node can include any appropriate data or data representation. The edge can describe any appropriate relationship between the data. In some implementations, the edge is marked or annotated so that it includes both the connection between the nodes and the descriptive information related to the connection. A specific node can be connected to one or more other nodes or itself through different edges to form an extended graph. For the sake of clarity, the graph based on the structure just described above is referred to as a knowledge graph here. In some implementations, a knowledge graph can be used to represent information and to provide information in a search.

[0110] Figure 3An illustrative knowledge graph portion 300 including nodes and edges according to some implementations of the present disclosure is shown. The illustrative knowledge graph portion 300 includes nodes 302, 304, 306, 308. The knowledge graph portion 300 includes an edge 310 for connecting node 302 and node 304. The knowledge graph portion 300 includes an edge 312 for connecting node 302 and node 306. The knowledge graph portion 300 includes an edge 314 for connecting node 304 and node 308. The knowledge graph portion 300 includes an edge 316 and an edge 318 for connecting node 302 and node 308. The knowledge graph 300 includes an edge 320 for connecting node 308 to itself. Each of the above groups of an edge and one or two different nodes may be referred to as a 3-tuple. As illustrated, node 302 is directly connected to three other nodes via an edge, and nodes 304 and 308 are directly connected to two other nodes via an edge. Node 306 is connected to only one other node by an edge, and in some implementations, node 306 is referred to as a terminal node. As illustrated, nodes 302 and 308 are connected by two edges, indicating that the relationship between nodes is defined by more than one attribute. As illustrated, node 308 is connected to itself by edge 320, indicating that the node can be related to itself. Although the illustrative knowledge graph 300 includes edges that are not marked as directed, it should be understood that each edge can be unidirectional or bidirectional. It should be understood that the example of the figure is only an example and any appropriate size or arrangement of nodes and edges can be used.

[0111] Typically, nodes in a knowledge graph can be grouped into several categories. Nodes can represent entity references, organizational data such as entity types and attributes, literal values, and relationship models between other nodes.

[0112] In some implementations, entity references, entity types, attributes, and other appropriate content may be created, defined, redefined, modified, or otherwise generated by any appropriate technology. For example, content may be generated by manual user input, by automatic response to user interaction, by importing data from an external source, by any other appropriate technology, or any combination thereof. For example, if a commonly used search for a term is not represented in the knowledge graph, one or more nodes may be added to represent the node. In another example, a user may manually add information and organizational structures.

[0113] The nodes of the knowledge graph can represent entities. An entity is a separate, unique, well-defined, and distinguishable thing or concept. For example, an entity can be a person, a place, an item, an idea, an abstract concept, a concrete element, other appropriate things, or any combination thereof. It should be understood that in some implementations, the knowledge graph contains entity references rather than the physical embodiment of the entity. For example, an entity can be the physical embodiment of George Washington, while the entity reference refers to the abstract concept of George Washington. In another example, the entity "New York City" refers to a physical city, and the knowledge graph uses the concept of a physical city as represented, for example, by an element in a data structure, the name of an entity, any other appropriate element, or any combination thereof. Where appropriate, based on the context, it should be understood that the term entity as used herein can correspond to an entity reference, and the term entity reference as used herein can correspond to an entity.

[0114] Nodes are unique in that no two nodes refer to the same thing or concept. Typically, an entity includes a thing or concept linguistically represented by a noun. For example, the color "blue", the city "San Francisco", and the imaginary animal "unicorn" can each be an entity. An entity reference typically refers to the concept of an entity. For example, the entity reference "New York City" refers to the physical city, and the knowledge graph uses the concept of the physical city as represented, for example, by an element in a data structure, the name of an entity, any other appropriate element, or any combination thereof.

[0115] Nodes representing organizational data may be included in the knowledge graph. These may be referred to as entity type nodes here. As used herein, entity type nodes may refer to nodes in the knowledge graph, and entity types may refer to concepts represented by entity type nodes. Entity type may be a defining feature of an entity. For example, entity type node Y may be connected to entity reference node X via an "is" edge or link discussed further below, so that the graph represents information "entity X is type Y". For example, entity reference node "George Washington" may be connected to entity type node "President". Entity reference nodes may be connected to multiple entity type nodes, such as "George Washington" may also be connected to entity type node "person" and entity type node "military commander". In another example, entity type node "city" may be connected to entity reference nodes "New New York City" and "San Francisco". In another example, although the concept "tall person" is not fully defined, such as a knowledge graph does not necessarily include a definition of "tall", the concept "tall person" may exist as an entity type node. In some implementations, the existence of entity type node "tall person" and other entity type nodes may be based on user interaction.

[0116] In some implementations, an entity type node may include or be connected to data related to: a list of attributes associated with the entity type node, the domain to which the entity type node belongs, a description, a value, any other appropriate information, or any combination thereof. A domain refers to a collection of related entity types. For example, the domain "films" may include, for example, entity types "actors", "directors", "filming locations", "movies", any other appropriate entity types, or any combination thereof. In some implementations, an entity reference is associated with types in more than one domain. For example, the entity reference node "Benjamin Franklin" may be connected to an entity type node "politician" in the domain "government" and an entity type node "inventor" in the domain "occupation".

[0117] In some implementations, the attributes associated with the entity reference node or entity type node can also be represented as nodes. For example, a node representing the attribute "population" or "location" can be connected to the entity type node "city". The combination and / or arrangement of entity types and their attributes is called a pattern. In some implementations, the pattern is stored in a table or other appropriate data structure associated with the entity type node. In some implementations, the knowledge graph can be customized or bootstrapped so that it includes specific nodes and edges for defining the concepts of nodes, edges, and the graph itself. For example, a knowledge graph can include an entity reference node "Knowledge Graph", which is connected to attribute nodes such as "having a node" and "having an edge" for describing the attributes of the knowledge graph.

[0118] In some implementations, specific values ​​called literals can be associated with specific entity references in terminal nodes through edges that are used to define relationships. Literals can refer to values ​​and / or strings of information. For example, literals can include dates, names, and / or numbers. In the example, the entity reference node "San Francisco" can be connected to a terminal node containing the literal "815000" through an edge labeled with the attribute "has population". In some implementations, terminal nodes can contain references or links to long text strings and other information stored in one or more documents outside the knowledge graph. In some implementations, literals are stored as nodes in the knowledge graph. In some implementations, literals are stored in the knowledge graph, but are not assigned unique identifying references as described below, and cannot be associated with multiple entity references. In some implementations, literal type nodes can define the type of literals such as "date / time", "number", "GPS coordinates".

[0119] In some implementations, the group of an edge and two nodes is called a triple. A triple represents a relationship between nodes, or in some implementations, represents a relationship between the node and itself. In some implementations, a higher-order relationship such as a quaternary or n-ary relationship is modeled, where n is an integer greater than 2. In some implementations, the information used to model the relationship is stored in a node, which can be called an intermediary node. In the example, the information "Person X donated stone tool Y to Museum Z" is stored in an intermediary node for connecting an entity reference node to X, Y, and Z, where each edge identifies the role of each correspondingly connected entity reference node. In some implementations, a knowledge graph may include information for distinguishing and disambiguating words and / or entities. As used herein, distinguishing refers to a many-to-one situation where multiple names are associated with a single entity. As used herein, disambiguation refers to a one-to-many situation where the same name is associated with multiple entities. In some implementations, a unique identification reference may be assigned to a node. In some implementations, a unique identification reference may be an alphanumeric string, a name, a number, a binary code, any other appropriate identifier, or any combination thereof. Uniquely identifying a reference may allow a search system to assign unique references to nodes having the same or similar textual identifiers. In some implementations, unique identifiers and other techniques are used in differentiation, disambiguation, or both.

[0120] In some implementations of differentiation, a node may be associated with multiple words or words and distinguished aliases associated with the same entity. For example, the words "George Washington", "George Washington", "President Washington", and "President George Washington" may all be associated with a single entity reference such as a node in a knowledge graph. This may provide differentiation and simplification of the knowledge graph. In some implementations of disambiguation, multiple nodes with the same or similar names are defined by their unique identification references, by associated nodes in the knowledge graph, by any other appropriate information, or by any combination thereof. For example, there are entity reference nodes related to the city "Philadelphia", entity reference nodes related to the movie "Philadelphia", and entity reference nodes related to the cream cheese brand "Philadelphia". Each of these nodes may have a unique identification reference for disambiguation within the knowledge graph, such as stored as a number. In some implementations, disambiguation in the knowledge graph is provided by connections and relationships between multiple nodes. For example, the city "New York" may be distinguished from the state government "New York" because the city is connected to the entity type "city" and the state government is connected to the entity type "state government". It should be understood that more complex relationships may also define nodes and disambiguate nodes. For example, a node may be defined by an associated entity type, other entity references connected to it by specific attributes, its name, any other appropriate information, or any combination thereof. These connections may be used in disambiguation, e.g., a node "Georgia" connected to a node "United States" may be understood to represent a state in the United States, while a node "Georgia" connected to nodes "Asia" and "Eastern Europe" may be understood to represent a country in Eastern Europe.

[0121] In some implementations, a node may include data for defining one or more attributes or be connected to the data. Attributes may define specific features of a node. The specific attributes of a node may depend on what the node represents. In some implementations, an entity reference node may include or be connected to: a unique identifier reference, a list of entity types associated with the node, a list of distinguishing aliases of the node, data associated with the entity reference, a text description of the entity reference, a link to the text description of the entity reference, other appropriate information, or any combination thereof. As described above, a node may contain references or links to long text strings and other information stored in one or more documents outside the knowledge graph. In some implementations, the storage technology may depend on specific information. For example, a unique identifier reference may be stored within a node, a short information string may be stored as text in a terminal node, and a long description of an entity may be stored in a linked external document through a reference in the knowledge graph. An edge in a knowledge graph may represent a semantic connection for defining a relationship between two nodes. An edge may represent a predicated connection such as "is", "has", "type is", "has an attribute", "has a value", any other appropriate statement, or any combination thereof. For example, an entity reference node for a particular person may be connected via an edge "date of birth" to a terminal node containing text for his or her particular date of birth. In some implementations, the attributes defined by the edge connections for entity references may be related to nodes connected to the type of entity reference. For example, the entity type node "movie" may be connected to the entity reference nodes "actor" and "director", and a particular movie may be connected to an entity reference node representing a particular actor via an edge attribute "has actor".

[0122] In some implementations, nodes and edges define the relationship between entity type nodes and their attributes, thereby defining a pattern. For example, an edge can connect an entity type node to a node associated with an attribute, which can be called an attribute node. This type of entity reference can be connected to a node used to define specific values ​​of those attributes. For example, the entity type node "person" can be connected to the attribute node "date of birth" and the node "height". In addition, the node "date of birth" can be connected to the text type node "date / time", indicating that the text associated with "date of birth" includes date / time information. The entity reference node "George Washington" connected to the entity type node "person" through the "is" edge can also be connected to the text "February 22, 1732" through the edge "has a date of birth". In some implementations, the entity reference node "George Washington" is connected to the "date of birth" attribute node. It should be understood that in some implementations, the same technology is used to model both the pattern and the data and store them in the knowledge graph. In this way, both the pattern and the data can be accessed by the same search technology. In some implementations, the pattern is stored in a separate table, chart, list, other data structure, or any combination thereof. It should also be understood that attributes may be modeled by nodes, edges, text, any other suitable data, or any combination thereof.

[0123] For example, the entity reference node "George Washington" can be connected to the entity type node for representing "person" through the "is" edge, thereby indicating the entity type of the entity reference, and can also be connected to the text "February 22, 1732" through the edge "has a date of birth", thereby defining the attributes of the entity reference. In this way, the knowledge graph defines both the entity type and the attribute associated with a specific entity reference by being connected to other nodes. In some implementations, "February 22, 1732" can be a node so that it is connected to other events that occurred on that date. In some implementations, the date can be further connected to a year node, a month node, and a day node. It should be understood that the information can be stored in any appropriate combination of text, nodes, terminal nodes, interconnected entity references, any other appropriate arrangement, or any combination thereof.

[0124] Figure 4An illustrative knowledge graph portion 400 according to some implementations of the present disclosure is shown. The knowledge graph portion 400 includes information related to the entity reference "George Washington" represented by the "George Washington" node 402. "George Washington" 402 is connected to the "President of the United States" entity type node 404 by an "is" edge 414 with semantic content "is", so that the 3-tuple defined by nodes 402 and 404 and edge 414 contains the information "George Washington is the President of the United States". Similarly, the information "Thomas Jefferson is the President of the United States" is represented by the tuple of the "Thomas Jefferson" node 410, the "is" edge 420, and the "President of the United States" node 404. The knowledge graph portion 400 includes the entity type node "person" 424 and the "President of the United States" node 404.

[0125] Types of people are defined in part by connections from "person" node 424. For example, type "person" is defined as having an attribute "date of birth" by node 430 and edge 432, and type "person" is defined as having an attribute "gender" by node 434 and edge 436. These relationships partially define the pattern associated with the entity type "person."

[0126] In the knowledge graph portion 400, it is shown that "George Washington" 402 has entity types "person" and "President of the United States", and is thus connected to nodes containing values ​​associated with those types. For example, the "George Washington" node 402 is connected to the "male" node 406 via the "has gender" edge 418, indicating that "George Washington" has the gender "male". In addition, the "male" node 406 can be connected to the "gender" node 434, indicating that "male is a gender". Similarly, the "George Washington" node 402 is connected to the "February 22, 1732" node 408 via the "has birth date" edge 416, indicating that "George Washington has a birth date of February 22, 1732". The "George Washington" node 402 can also be connected to the "1789" node 428 via the "has inauguration date" edge 430.

[0127] The knowledge graph portion 400 also includes “Thomas Jefferson” 410, which is connected to the entity type “President of the United States” node 404 via an “is” edge 420 and to the entity type “Person” node 424 via an “is” edge 428. Thus, the knowledge graph portion 400 indicates that “Thomas Jefferson” has entity types “President of the United States” and “Person”. In some implementations, the “Thomas Jefferson” node 410 is connected to the entity type “President of the United States” node 404 via an “is” edge 428. Figure 4 The nodes related to their date of birth, gender, and date of employment, which are not shown in the figure, are connected.

[0128] It should be understood that the knowledge graph part 400 is only an example and it can include nodes and edges not shown.For example, "President of the United States" node 404 can be connected to all Presidents of the United States. "President of the United States" node 404 can also be connected to the relevant attributes of the entity type such as "4 years" duration of term, such as "2 terms" term limit, such as "Washington DC" office location, any other appropriate data or its arbitrary combination.For example, "President of the United States" node 404 is connected to "date of inauguration" node 438 by "having attribute" edge 440, which partially defines the pattern of type "President of the United States".Similarly, "Thomas Jefferson" node 410 can be connected to the node of any appropriate number, and the node contains further information related to his entity type node "President of the United States" and "people" described, and "Thomas Jefferson" node 410 can be connected to other entity type nodes not shown such as "inventor", "vice president" and "writer".In a further example, "people" node 424 can be connected to all entity references with type "people" in the knowledge graph. In a further example, the "1789" node 428 may be connected to all events in the knowledge graph that have the attribute year "1789". The "1789" node 428 is unique to the year 1789 and is referenced by its unique identifier to eliminate confusion with, for example, Figure 4 In some implementations, the "1789" node 428 is connected to the entity type node "year".

[0129] Figure 5Another illustrative knowledge graph portion 300 according to some implementations of the present disclosure is shown. The knowledge graph portion 500 includes a "California" node 502, which may also be associated with a distinguishing alias such as, for example, "CA," "California," "Golden State," any other suitable distinguishing alias, or any combination thereof. In some implementations, these distinctions are stored in the "California" node 502. California is connected to the "US State Government" entity type node 506 via an "Is" edge 504. The "New York" node 510 and the "Texas" node 514 are also connected to the "US State Government" node 506 via "Is" edges 508 and 512, respectively. The "California" node 502 is connected to the "Sacramento" node 522 via a "Has Capital City" edge 520, indicating information that "California has a capital city of Sacramento." The Sacramento node 522 is further connected to the "City" entity type node 526 via an "Is" edge 524. Similarly, the "Texas" node 514 is connected to the "Houston" node 528 via the "Has City" edge 520, which is further connected to the "City" entity type node 526 via the "Is" edge 540. Similarly, the "Texas" node 514 is connected to the "Houston" node 528 via the "Has City" edge 520, which is further connected to the "City" entity type node 526 via the "Is" edge 540. The "California" node 502 is connected to the node 518 containing the literal value "37,691,612" via the "Has Population" 516. In the example, the specific value "37,691,,612" is periodically automatically updated by the knowledge graph based on an external URL or other source of data. The knowledge graph portion 500 may include other nodes not shown. For example, the "US State Government" entity type node 506 may be connected to nodes for defining attributes of the entity type such as "Population" and "Capital City." These entity type attribute relationships can be used to define other relationships in the knowledge graph portion 500, such as the "has population" edge 516 connecting the entity reference node "California" 516 to the terminal node 518 containing text defining the population of California. It should be understood that although in the following Figure 4 The knowledge graph part 400 and Figure 5 The knowledge graph portion 500 of shows a portion of a knowledge graph, but all information can be contained within a single graph, and the options described here are merely examples. In some implementations, separate knowledge graphs are saved for different corresponding domains, for different corresponding entity types, or according to any other suitable demarcation feature. In some implementations, separate knowledge graphs are saved according to size constraints. In some implementations, a single knowledge graph is saved for all entity references and entity types.

[0130] The knowledge graph can be implemented using any suitable software structure. In an example, the knowledge graph is implemented using an object-oriented structure in which each node is an object with associated functions and variables. In this context, an edge can be an object with associated functions and variables. In some implementations, the data contained in the knowledge graph, the nodes pointing to the knowledge graph, or both are stored in any suitable one or more data repositories located on one or more servers in one or more geographical locations coupled by any suitable network architecture.

[0131] Figure 6-8 An illustrative sequential screenshot of a user interface based on an illustrative query [nuclear reactors near volcanoes] is shown.

[0132] Figure 6 is an illustrative first screenshot 600 of a user interface of a search system according to some implementations of the present disclosure. The screenshot 600 includes a map 610 of a portion of the United States and Canada with nuclear reactors 602 labeled by hollow circles. A first text query [nuclear reactors] is shown in a text box 620. Figure 6 , 7 , 8, "nuclear reactor" is the first entity type of the query. Nuclear reactor 602 includes a collection of entity references stored in a data structure such as a knowledge graph, wherein each corresponding entity reference is associated with a location illustrated by indicator 630 and is associated with a minimum power capacity indicated by indicator 640. Indicators 630 and 640 represent display-related settings such as display settings or filter settings. For example, the displayed nuclear reactor 602 can be filtered by location, capacity, other metrics, or a combination thereof, such as shown by filter menu 660. The graphical representation can also indicate one or more criteria, such as the size of the hollow circle can be adjusted by the nuclear reactor capacity. In response to the query [nuclear reactor], the search system can determine geographic information for each nuclear reactor in order to annotate the map 610 with a hollow circle. For example, the geographic information includes longitude and latitude values ​​contained in the geographic attributes associated with each considered nuclear reactor entity reference in the knowledge graph. In another example, the knowledge graph may include geographic information of the entity referenced here.

[0133] Figure 7is an illustrative second screenshot 700 of a user interface for a search system according to some implementations of the present disclosure. Screenshot 700 follows screenshot 600 and shows an additional query [near volcano] entered in "near" text box 650. In some implementations, menu 652 may be used to allow the user to select a desired relationship. For example, in addition to "near," menu 652 may include "far away," "between X and Y miles," or any other suitable selectable relationship options. Volcano 710 is shown as a black dot on map 610. In the illustrated Figure 6 , 7 , 8, "volcano" indicates a second entity type. The search system can interpret the query criterion "nearby" to mean "within a predetermined distance" based on the spatial relationship of the first and second entity types, for example, entities are all places that have a location. Therefore, each volcano 710 shown in screenshot 700 is within a predetermined distance of one or more nuclear reactors 610, for example, unspecified but greater than 200 miles. The volcanoes and nuclear reactors shown in screenshot 700 satisfy the query [nuclear reactor and near volcano].

[0134] Figure 8 is an illustrative third screenshot 800 of a user interface of a search system according to some implementations of the present disclosure. Screenshot 800 follows screenshot 700 and shows an additional query [within 100 miles] as entered using filter menu 660. Indicator 820 displays the criterion [within 100 miles] along with a user-adjustable sliding bar that can be used to filter the displayed volcano results. Thus, only volcanoes within 100 miles of a nuclear reactor are displayed. The volcanoes and nuclear reactors shown in screenshot 800 satisfy the query [nuclear reactors and volcanoes within 100 miles]. Figure 6 , 7 Screen shots 600, 700, and 800 of , respectively, illustrate user interactions to refine one result set by another result set, and may allow the user to define one or more criteria, such as distance in a map view. Thus, the search system may apply other refinements to narrow the search results and identify the best entity reference pairs or larger entity reference groups based on, for example, user input, data in the knowledge graph, shared attributes, aggregated user selections, any other information, or any combination thereof.

[0135] Fig. 99 is another illustrative screenshot 900 of a user interface of a search system according to some implementations of the present disclosure. Screenshot 900 includes a map 910 of a portion of the United States and Canada with nuclear reactors 902 labeled by hollow circles and volcanoes 904 labeled by black dots. The original text query [nuclear reactors within 100 miles of volcanoes] is shown in text box 920. The volcanoes and nuclear reactors shown in screenshot 900 satisfy the query [nuclear reactors within 100 miles of volcanoes]. Fig. 9 In the example of , "nuclear reactor" indicates a first entity type, "volcano" indicates a second entity type, and "within 100 miles" indicates a criterion. It should be noted that the screenshot 900 shows the filtering results of the first type and the second type of entity references, while Figure 8 Screen shot 800 of shows entity references of the first type with only entity references of the second type filtered. According to the present disclosure, the search system may filter entity references of one, two, or more than two types. In addition, according to the present disclosure, the search system may display entity references of one, two, or more than two types. For example, referring to Fig. 9 , the search system may optionally display only nuclear reactors 902 that meet criteria not shown.

[0136] Such as Figure 6 , 7 Similar techniques to those described in the context of , 8, and 9 are applied to entity types with temporal relationships. Fig.10is an illustrative screenshot 1000 of a user interface of a search system showing a timeline 1010 according to some implementations of the present disclosure. The screenshot 1000 includes a timeline 1010 with a date range from 1986 to 2015 with bankruptcy filings 1002 marked by hollow circles and the U.S. financial crisis 1004 marked by shaded boxes. Four U.S. financial crises and sixteen bankruptcy filings are displayed in the screenshot 1000, satisfying the query [U.S. bankruptcy filings of companies with assets greater than $30 billion within 2 years of a U.S. financial crisis]. Thus, entity references of a first entity type (bankruptcy filings in the illustrated example) are filtered by company size and are close to financial crises, while entity references of a second type (U.S. financial crises in the illustrated example) are substantially unfiltered. In some implementations, the search system may determine which type of entity references to filter and / or display, if any. For example, the search system may display only the bankruptcy filings 1002 on the timeline 1010, without displaying the U.S. financial crisis 1004. The search system determines which entity references to filter and / or display can be based on a query, a predetermined setting, the type of entity reference under consideration, any other appropriate information, or a combination thereof. Indicators 1030 and 1040 respectively display filtering criteria, which can be adjusted by user interaction with corresponding sliders. The determination of the indicators to be displayed can be based on a query, a predetermined setting, the type of entity reference under consideration, any other appropriate information, or a combination thereof. For example, if the user adjusts the slider of indicator 1030 to a filter setting of [1 year], the search system can remove the "Refco" bankruptcy filing from screenshot 1000 because it is not within a year of the U.S. financial crisis.

[0137] Fig.11 1100 is a flowchart of illustrative steps for identifying result entity references based on a combined query according to some implementations of the present disclosure. According to some implementations of the present disclosure, the illustrative steps of flowchart 1100 may be rearranged, omitted, or otherwise modified. It should be understood that although flowchart 1100 illustrates steps for a first entity type and a second entity type, in some implementations the search system processes any suitable number of entity types.

[0138] Step 1102 is that the search system identifies the first entity type, the second entity type, and the relationship. In some implementations, the first entity type, the second entity type, and the relationship thereof can be identified from the query. In some cases, both the first entity type and the second entity type are entity types with location attributes. For example, the entity type can be a restaurant, a bank, a building, an office, a bar, a cafe, a gas station, an entertainment venue, a department store, a stadium, a library, a national park, a lake, a nuclear reactor, a volcano, any other appropriate entity type with a location attribute, which can be filtered or restricted by any appropriate criteria, such as a restaurant limited to Japanese restaurants, or any combination thereof. In some cases, the first entity type and the second entity type are entity types with a time attribute. For example, the entity type can be a birthday, a death, a life span, a war, a financial crisis, an inauguration, a term of office, an application such as a bankruptcy application, a broadcast date such as a television program, any other appropriate event, any other appropriate entity type with a time attribute that is filtered or restricted by, for example, the birthday of the President of the United States, or any combination thereof. Relationships between entities can include any suitable comparable attribute values ​​such as, for example, building height, animal genus / species, car specifications, spouses' incomes, any other attributes that can be compared between entities, or any combination thereof. For example, a query such as [husbands and wives with an income gap of more than $1 million] can be resolved by comparing the attribute value of "income". In yet another example, a query such as [close buildings in New York City with a height difference of at least 500 feet] can be resolved by comparing the attribute value of building height. In the aforementioned example, the search system can also compare the attribute value of location based on the "close" criterion, thereby providing search results based on two sets of attribute values, such as building height and location.

[0139] Step 1104 is a search system for identifying nodes of the knowledge graph corresponding to entity references of the first entity type and entity references of the second entity type. For example, the first entity type may be "US financial crisis", and the search system may identify nodes in the knowledge graph corresponding to Black Monday, the savings and loan crisis, the Internet bubble, and the 2007 housing bubble.

[0140] Step 1106 is the search system determining attribute values ​​for each entity reference of the first entity type and each entity reference of the second entity type. For example, the search system can determine the attribute values ​​by searching the knowledge graph and accessing the relevant nodes. In some implementations, the attribute values ​​may include location information such as an address, a longitude / latitude value, a relative position of a reference to another entity, any other appropriate location information, or any combination thereof. In some implementations, the attributes may include time information such as a date, a time of day, a year, a century, a relative time interval of a reference to another entity, any other appropriate time information, or any combination thereof. The attribute values ​​may be numbers, text, alphanumeric strings, or have any other format.

[0141] Step 1108 is the search system comparing the attribute values ​​of each entity of the first entity type with the attribute values ​​of each entity of the second entity type. For example, for N entity references of the first type and M entity references of the second type, the search system can perform N×M comparisons. In some implementations, the search system can compare entity references by determining the difference in their corresponding attribute values. For example, for two entity references with location attribute values, the search system can determine the relative distance between the entity references in any appropriate units. In another example, for two entity references with time attribute values, the search system can determine the relative time interval between the entity references in any appropriate units. It should be understood that the attribute values ​​identified and compared by the search system can depend on the combined query and that a particular entity reference may have associated attributes that include both location and time information. In some implementations, the search system can compare entity references based on information such as Figure 2 , to bypass some N×M comparisons. For example, if the reference table indicates that for a particular entity reference of a first entity type, there are no entity references of a second entity type that meet the criteria, then the search system does not have to perform the comparison, which therefore reduces computation time.

[0142] Step 1110 is the search system identifying one or more result entity references based on the comparison of step 1108. The result entity references include a subset of the set entity references of the first entity type and the second entity type. In some implementations, the result entity references may include only entity references of the first entity type or the second entity type. For example, in response to the combined query [Bank of America near Japanese restaurants], the search system may select only "Bank of America" ​​as the result entity reference.

[0143] In some implementations, the search system may select result entity references based on criteria. The criteria may be explicitly included in the query, may be implied by the query, or may be determined by the search system based on a predetermined setting.

[0144] In an illustrative example, a search system identifies result entity references that have location attributes. The search system computes latitude and longitude for all geographic entity references in a knowledge graph. Some entity references themselves have geographic attributes such as, for example, Mount Everest, while others do not. For example, the entity reference "Google New York Office" may have an attribute called "location," which may be a building. The building itself may contain geographic location attributes such as latitude and longitude. The search system generates a data structure for each entity reference in the knowledge graph, the data structure including an identifier such as an identification number corresponding to each respective entity reference and a location such as latitude and longitude. The identifier is an entity unique ID used to index entity data in the data structure. In some implementations, a hash map is used to store the identification and location data, so that given an ID, the search system can typically retrieve the corresponding latitude and longitude in 0(1) time. When the search system receives a query, the system recognizes the two entity types in the query and identifies entity references of the two entity types. Using the ID and location, the search system identifies the latitude and longitude for all entity references. By calculating the distance between each entity reference of the first entity type and each entity reference of the second entity type, the search system determines which pairs of entities are close to each other based on a distance criterion, for example, specified by a user or predetermined by the search system. For entity references with time attributes, the search system calculates the time difference and determines which pairs of entities are close to each other based on a time criterion, for example, specified by a user or predetermined by the search system. In some implementations, the time complexity of the disclosed algorithm is O(N*M), where N is the number of entity references of the first entity type and M is the number of entity references of the second entity type.

[0145] Fig.121200 is a flowchart of illustrative steps for storing data in a data structure for use with a combined query according to some implementations of the present disclosure. In some cases, the search system may identify many entity references in each entity type, such as 10,000 entity references or more, resulting in greater time complexity. In this case, the search system will need to run 10,000*10,000 (thus 100,000,000) distance calculations or more, and then compare them to each other to find the best pair. In order to reduce computation time, a pre-generated data structure can be used. In some implementations, the illustrative steps of flowchart 1200 can be performed without targeting a specific query such as, for example, when executed as an offline algorithm. The search system can generate a table offline and identify results relatively quickly. In some implementations, the search system can limit only one entity type in the query. For example, with a reference type "airport", the data structure can include airports in California because the data is established before responding to a specific query. According to some implementations of the present disclosure, the illustrative steps of flowchart 1200 can be rearranged, omitted, or otherwise modified. It will be appreciated that although flowchart 1200 illustrates steps for a first entity type and a second entity type, in some implementations the search system processes any suitable number of entity types.

[0146] Step 1202 is that the search system identifies entity references of the first entity type. The search system identifies entity references of the first entity type by searching the knowledge graph. The first entity type may be determined by the search system or by a user, for example.

[0147] Step 1204 is that the search system determines an attribute value for each entity reference of the first entity type identified in step 1202. The attribute value may include location information, time information, any other appropriate information, or any combination thereof. The attribute may be determined by the search system or by the user, for example.

[0148] Step 1206 is the search system identifying entity references of multiple second entity types. The second entity type can be determined by the search system or by the user. In some implementations, the multiple second entity types are determined based on search history such as the most popular entity type searched. In some implementations, the multiple second entity types are determined based on their actual relationship with the first entity type. For example, for the first entity type "airport", the multiple second entity types may include "hotel", "restaurant", "tourist attraction", and "conference and tourism center".

[0149] Step 1208 is the search system determining the difference between the attribute value of each entity reference of the first entity type and the attribute value of each entity reference of the second entity type. The difference can be a distance, a time interval, any other suitable difference, or any combination thereof.

[0150] Step 1210 is the search system selecting a target entity reference of the second entity type for each entity reference of the first entity type having the minimum difference determined in step 1208. In some implementations, the search system identifies these closest pairs by entity reference, entity type, or both.

[0151] Step 1212 is the search system storing data representing each target entity reference of the second entity type in a data structure. In some implementations, the target entity references are indexed by the corresponding entity reference of the first entity type. In some implementations, only the differences between the attribute values ​​are stored, rather than the entity identifiers themselves. For example, Figure 2 As shown, the data structure may include a difference value, an entity identifier, or both.

[0152] Fig.13 1300 are illustrative steps for displaying at least some result entity references based on a combined query according to some implementations of the present disclosure. According to some implementations of the present disclosure, the illustrative steps of flowchart 1300 may be rearranged, omitted, or otherwise modified. It will be understood that although flowchart 1300 illustrates steps for a first entity type and a second entity type, in some implementations the search system processes any suitable number of entity types.

[0153] Step 1302 is the search system receiving user input indicating a first entity type, a second entity type, and a relationship defining a criterion. The user input may include keystrokes such as a typed text string, menu selections, language input into a microphone, any other suitable input from the user to the search system, or any combination thereof. For example, the user input may be in the form of a typed query. In another example, the user input may be in the form of a menu selection and corresponding filter settings. The criterion may be explicit, such as [between 10 and 20 miles]; or implicit, such as [nearby], in which case the search system can determine the meaning of the "nearby" criterion.

[0154] Step 1304 is the search system identifying pairs of entity references of the first and second entity types that meet the criteria. The pairs include entity references of the first entity type and entity references of the second entity type, and need not be exclusive. For example, a particular entity of the first entity type may be included in multiple pairs with respective corresponding entity references of the second entity type. The identified pairs include comparing corresponding attribute values ​​of the entity references of the first and second entity types to determine a difference, and then comparing the difference to the criteria.

[0155] Step 1306 is causing the search system to display a representation of the entity references from the pair identified in step 1304. The search system may be caused to display entity references of the first entity type, entity references of the second entity type, or both. The representation may depend on the entity references, entity types, differences, user input, any other suitable information, or any combination thereof. For example, the search system may display a map of the relevant scale and information such as Figure 6 , 7 , 8, and 9. In another example, the search system can display a timeline of related scales and a timeline of related scales. Fig.10 The pairs shown include time annotations of differences in time intervals. In some implementations, the search system may also be caused to display one or more filters, menus, text boxes, or other displayed features that allow user interaction. The displayed features may allow the user to, for example, filter the set of entity reference pairs, adjust the display, adjust the query, adjust the criteria, provide any other input, or any combination thereof.

[0156] The following figures describe illustrative computer systems that may be used in some implementations of the present disclosure. It should be understood that the knowledge graph and associated techniques may be implemented on any suitable processor or combination of processors, which may be included in one or more computers, for example.

[0157] Fig.14 An illustrative computer system 1400 according to some implementations of the present disclosure is shown. System 1400 may include one or more user devices 1402. In some implementations, user device 1402 may include a smart phone, a tablet computer, a desktop computer, a laptop computer, a personal digital assistant, a portable audio player, a portable video player, a mobile gaming device, other suitable user devices capable of providing content, or any combination thereof. In some implementations, the search system is implemented using a computer system such as system 1400 or a portion thereof.

[0158] User device 1402 may be coupled to network 1404 directly, through connection 1406, through wireless repeater 1410, through any other suitable means of coupling to network 1404, or through any combination thereof. Network 1404 may include the Internet, a discrete network of computers and servers, a local network, a public intranet, a private intranet, other coupled computing systems, or any combination thereof.

[0159] User equipment 1402 may be coupled to network 1404 via wired connection 1406. Connection 1406 may include Ethernet hardware, coaxial cable hardware, DSL hardware, T-1 hardware, fiber optic hardware, analog phone line hardware, any other suitable wired hardware capable of communication, or any combination thereof. Connection 1406 may include transmission technology including TCP / IP transmission technology, IEEE 802 transmission technology, Ethernet transmission technology, DSL transmission technology, fiber optic transmission technology, ITU-T transmission technology, any other suitable transmission technology, or any combination thereof.

[0160] The user device 1402 may be wirelessly coupled to the network 1404 via a wireless connection 1408. In some implementations, the wireless repeater 1410 receives information transmitted from the user device 1402 via the wireless connection 1408 and communicates it with the network 1404 via the connection 1412. The wireless repeater 1410 receives information from the network 1404 via the connection 1412 and communicates it with the user device 1402 via the wireless connection 1408. In some implementations, the wireless connection 1408 may include a wireless transmission technology including cellular telephone transmission technology, code division multiple access or CDMA transmission technology, global system for mobile communications or GSM transmission technology, general packet radio service or GPRS transmission technology, satellite transmission technology, infrared transmission technology, Bluetooth transmission technology, Wi-Fi transmission technology, WiMax transmission technology, any other suitable transmission technology, or any combination thereof.

[0161] Connection 1412 may include Ethernet hardware, coaxial cable hardware, DSL hardware, T1 hardware, fiber optic hardware, analog telephone line hardware, wireless hardware, any other suitable hardware capable of communication, or any combination thereof. Connection 1412 may include wired transmission technology, including TCP / IP transmission technology, IEEE 802 transmission technology, Ethernet transmission technology, DSL transmission technology, fiber optic transmission technology, ITU-T transmission technology, any other suitable transmission technology, or any combination thereof. Connection 1412 may include wireless transmission technology, including cellular telephone transmission technology, code division multiple access or CDMA transmission technology, global system for mobile communications or GSM transmission technology, general packet radio service or GPRS transmission technology, satellite transmission technology, infrared transmission technology, Bluetooth transmission technology, Wi-Fi transmission technology, WiMax transmission technology, any other suitable transmission technology, or any combination thereof.

[0162] Wireless repeater 1410 may include any number of cellular telephone transceivers, network routers, network switches, communication satellites, any other devices for communicating information from user device 1402 to network 1404, or any combination thereof. It should be understood that the arrangement of connection 1406, wireless connection 1408, and connection 1412 is merely illustrative, and system 1400 may include any suitable number of any suitable devices for coupling user devices 1402 to network 1404. It should also be understood that any user device 1402 may be communicatively coupled to any user device, a remote server, a local server, any other suitable processing device, or any combination thereof, and may be coupled using any suitable techniques as described above.

[0163] In some implementations, any appropriate number of remote servers 1414, 1416, 1418, 1420 may be coupled to the network 1404. The remote servers may be general purpose, dedicated, or any combination thereof. One or more search engine servers 1422 may be coupled to the network 1404. In some implementations, the search engine server 1422 may include a knowledge graph, may include a processing device configured to access the knowledge graph, may include a processing device configured to receive a search query related to the knowledge graph, may include any other appropriate information or device, or any combination thereof. One or more database servers 1424 may be coupled to the network 1404. In some implementations, the database server 1424 may store the knowledge graph. In some implementations, in the presence of more than one knowledge graph, the more than one knowledge graph may be included in the database server 1424, may be distributed on any appropriate number of database servers and general purpose servers, or any combination thereof, by any appropriate technology. It should also be understood that the search system may use any appropriate number of general purpose, dedicated, storage, processing, search, any other appropriate servers, or any combination thereof.

[0164] Fig.15 According to some implementations of the present disclosure Fig.14 1502. User device 1402 may include input / output devices 1502 and processing device 1504. Input / output devices 1502 may include display 1506, touch screen 1508, buttons 1510, accelerometer 1512, global positioning system (often abbreviated as GPS) receiver 1536, camera 1538, keyboard 1540, mouse 1542, and audio device 1534 including speaker 1514 and microphone 1516. In some implementations, Fig.15The devices described in the figure may represent devices included in a smart phone user device. It should be understood that the specific devices included in the illustrative computer system may depend on the type of user device. For example, the input / output devices 1502 of a desktop computer may include a keyboard 1540 and a mouse 1542 and the accelerometer 1512 and GPS receiver 1536 may be omitted. It should be understood that the user device 1402 may omit any appropriately described elements and may include devices not shown such as a media drive, a data storage, a communication device, a display device, a processing device, any other appropriate device, or any combination thereof.

[0165] In some implementations, the display 1506 may include a liquid crystal display, a light emitting diode display, an organic light emitting diode display, an amorphous organic light emitting diode display, a plasma display, a cathode ray tube display, a projection display, any other suitable display capable of displaying content, or any combination thereof. The display 1506 may be controlled by the display controller 1518 or the processor 1524 in the processing device 1504, by a processing device inside the display 1506, by other control devices, or by any combination thereof. In some implementations, the display 1506 may display data from a knowledge graph.

[0166] The touch screen 1508 may include a sensor capable of sensing pressure input, capacitive input, resistive input, piezoelectric input, optical input, acoustic input, any other suitable input, or any combination thereof. The touch screen 1508 may be capable of receiving touch-based gestures. The received gestures may include information related to one or more locations on the surface of the touch screen 1508, the pressure of the gesture, the speed of the gesture, the duration of the gesture, the direction of the path of the tracked gesture on its surface, the action of the device related to the gesture, other suitable information related to the gesture, or any combination thereof. In some implementations, the touch screen 1508 may be optically transparent and located above or below the display 1506. The touch screen 1508 may be coupled to or controlled by the display controller 1518, the sensor controller 1520, the processor 1524, any other suitable controller, or any combination thereof. In some implementations, the touch screen 1508 may include a virtual keyboard capable of receiving, for example, a search query for identifying data in a knowledge graph.

[0167] In some embodiments, gestures received by touch screen 1508 may cause the corresponding display elements to be displayed substantially simultaneously, i.e., immediately after or with a short delay, via display 1506. For example, when the gesture is a finger or stylus moving along the surface of touch screen 1508, the search system may cause a visible line of any suitable thickness, color, or pattern indicating the path of the gesture to be displayed on display 1506. In some implementations, for example, a mouse pointer displayed on a display screen may be used to completely or partially replace the functionality of a desktop computer using a mouse, touch screen, or the like.

[0168] The button 1510 may be one or more electromechanical button mechanisms, sliding mechanisms, switch mechanisms, rocker mechanisms, toggle mechanisms, other suitable mechanisms, or any combination thereof. The button 1510 may be included in the touch screen 1508 as a predetermined area of ​​the touch screen, such as a soft key. The button 1510 may be included in the touch screen 1508 as an area of ​​the touch screen defined by the search system and indicated by the display 1506. Activating the button 1510 may send a signal to the sensor controller 1520, the processor 1524, the display controller 1520, any other suitable processing device, or any combination thereof. Activating the button 1510 may include receiving a push gesture, a slide gesture, a touch gesture, a press gesture, a time-based gesture (e.g., based on the duration of the push), any other suitable gesture, or any combination thereof from the user.

[0169] The accelerometer 1512 may be capable of receiving information related to motion characteristics, acceleration characteristics, orientation characteristics, tilt characteristics, and other suitable characteristics, or any combination thereof, of the user device 1402. The accelerometer 1512 may be a mechanical device, a micro-electromechanical device, a nano-electromechanical device, a solid-state device, any other suitable sensing device, or any combination thereof. In some implementations, the accelerometer 1512 may be a 3-axis piezoelectric micro-electromechanical integrated circuit configured to sense acceleration, direction, or other suitable characteristics by sensing changes in capacitance of an internal structure. The accelerometer 1512 may be coupled to the touch screen 1508 so that the processing device 1504 uses, at least in part, information about gestures received by the accelerometer 1512 to interpret the gestures.

[0170] A global positioning system (commonly abbreviated as GPS) receiver 1536 may be capable of receiving signals from global positioning satellites. In some implementations, a GPS receiver 1536 may receive information from one or more satellites orbiting the earth, including information about time, orbit, and other information related to the satellites. This information may be used to calculate the position of the user device 1402 on the surface of the earth. The GPS receiver 1536 may include a barometer that is not shown to improve the accuracy of the position. The GPS receiver 1536 may receive information related to the position of the user device 1402 from other wired and wireless communication sources. For example, the identification and location of a nearby cellular phone tower may be used instead of GPS data or in addition to GPS data to determine the position of the user device 1402.

[0171] Camera 1538 may include one or more sensors to detect light. In some implementations, camera 1538 may receive video images, still images, or both. Camera 1538 may include a charge coupled device, a complementary metal oxide semiconductor sensor, a photocell sensor, an IR sensor, any other suitable sensor, or any combination thereof. In some implementations, camera 1538 may include a device such as a light emitting diode that can generate light to illuminate an object. Camera 1538 may communicate information captured by one or more sensors to sensor controller 1520, processor 1524, any other suitable device, or any combination thereof. Camera 1538 may include lenses, filters, and other suitable optical devices. It should be understood that user device 1402 may include any suitable number of cameras 1538.

[0172] The audio device 1534 may include a sensor and a processing device for receiving and transmitting information using sound waves or pressure waves. The speaker 1514 may include a device for generating sound waves in response to a signal. In some implementations, the speaker 1514 may include an electroacoustic transducer in which an electromagnet is coupled to a diaphragm to generate sound waves in response to an electrical signal. The microphone 1516 may include an electroacoustic device to convert an acoustic signal into an electrical signal. In some implementations, a capacitive microphone may use a diaphragm as part of a capacitor so that the sound waves cause a change in the capacitance of the device, which may be used as an input signal through the user device 1402.

[0173] Speaker 1514 and microphone 1516 may be included within user device 1402, may be remote devices coupled to user device 1402 via any suitable wired or wireless connection, or may be any combination thereof.

[0174] The speaker 1514 and microphone 1516 of the audio device 1534 may be coupled to an audio controller 1522 in the processing device 1504. The controller may send and receive signals from the audio device 1534 and perform pre-processing and filtering steps prior to transmitting signals related to the input signals to the processor 1524. The speaker 1514 and microphone 1516 may be coupled directly to the processor 1524. The connection from the audio device 1534 to the processing device 1504 may be wired, wireless, other suitable arrangement for communicating information, or any combination thereof.

[0175] The processing device 1504 of the user device 1402 may include a display controller 1518 , a sensor controller 1520 , an audio controller 1522 , a processor 1524 , a memory 1526 , a communication controller 1528 , and a power supply 1532 .

[0176] The processor 1524 may include circuitry for interpreting signals input to the user device 1402 from, for example, the touch screen 1508 and the microphone 1516. The processor 1524 may include circuitry for controlling output to the display 1506 and the speaker 1514. The processor 1524 may include circuitry for executing instructions of a computer program. In some implementations, the processor 1524 may be based on an integrated electronic circuit capable of executing instructions of a computer program and include multiple inputs and outputs.

[0177] Processor 1524 may be coupled to memory 1526. Memory 1526 may include random access memory (often abbreviated as RAM), flash memory, programmable read-only memory (often abbreviated as PROM), erasable programmable read-only memory (often abbreviated as EPROM), a magnetic hard drive, a magnetic tape cassette, a magnetic floppy disk, an optical CD-ROM disc, a CD-R disc, a CD-RW disc, a DVD disc, a DVD+R disc, a DVD-R disc, any other suitable storage medium, or any combination thereof.

[0178] As described above, the functionality of the display controller 1518, the sensor controller 1520, and the audio controller 1522 may be implemented in whole or in part as discrete components in the user device 1402, in whole or in part integrated into the processor 1424, in part or in whole combined into a combined control element, or any combination thereof.

[0179] The communication controller 1528 can be coupled to the processor 1524 of the user device 1402. In some implementations, the communication controller 1528 can communicate radio frequency signals using the antenna 1530. In some implementations, the communication controller 1528 can communicate signals using a wired connection not shown. The wired and wireless communications communicated by the communication controller 1528 can use Ethernet, amplitude modulation, frequency modulation, bit stream, code division multiple access (commonly abbreviated as CDMA), global system for mobile communications (commonly abbreviated as GSM), general packet radio service (commonly abbreviated as GPRS), satellite, infrared, Bluetooth, Wi-Fi, WiMax, any other suitable communication configuration, or any combination thereof. The functionality of the communication controller 1528 can be implemented in whole or in part as a discrete component in the user device 1402, included in whole or in part in the processor 1524, or any combination thereof. In some implementations, the communication controller 1528 can be coupled to a processor such as a wireless communication device. Fig.14 Such a network 1404 communicates and can receive data from a storage device such as a Fig.14 Information of the knowledge graph in database 1424.

[0180] The power supply 1532 can be coupled to the processor 1524 and other components of the user device 1402. The power supply 1532 can include a lithium ion polymer battery, a lithium ion battery, a NiMH battery, an alkaline battery, a lead acid battery, a fuel cell, a solar panel, a thermoelectric generator, any other suitable power source, or any combination thereof. The power supply 1532 can include a hard-wired connection to the power supply, and can include an electrical device for converting the voltage, frequency, and phase of the power supply input into the appropriate power for the user device 1402. In some implementations of the power supply 1532, a wall socket can provide 120V, 60Hz alternating current, usually abbreviated as AC. The circuits of transformers, resistors, inductors, capacitors, transistors, and other suitable electronic components included in the power supply 1132 can convert the 120V AC from the wall socket power into 5V of 0Hz, such as direct current. In some implementations of the power supply 1532, a lithium ion battery including a cathode based on lithium metal oxide and an anode based on graphite can provide 3.7V to the components of the user device 1402. Power supply 1532 may be fully or partially integrated into user device 1402 or may function as a standalone device. Power supply 1532 may directly power user device 1402, may power user device 1402 by charging a battery, may provide power in any other suitable manner, or any combination thereof.

[0181] The foregoing is merely an illustration of the principles of the present disclosure and various modifications may be made to those skilled in the art without departing from the scope of the present disclosure. The above implementations are presented for illustrative purposes rather than for limiting purposes. The present disclosure may also take many forms other than those explicitly described herein. Therefore, it is emphasized that the present disclosure is not limited to the methods, systems, and devices explicitly disclosed, but is intended to include modifications and variations thereto within the spirit of the appended claims.

Claims

1. A method implemented by one or more processors, the method comprising: Determine based on combined query: The first entity type, The second entity type, and Distance standards; Identify from the data structure: a plurality of first candidate entity references of the first entity type, and a second entity reference of the second entity type; Determine from the data structure an attribute value including a corresponding position of each of the plurality of first candidate entity references and the second entity reference; comparing the attribute value of each of the first candidate entity references with the attribute value of the second entity reference to determine a corresponding distance between each of the first candidate entity references and the second entity reference; determining one or more result first entity references from the first candidate entity references based on the corresponding distance and the distance criterion, and In response to receiving the combined query and displaying it as a mark on a map of a graphical user interface: the second entity references, and At least one of the one or more result first entity references.

2. The method according to claim 1, wherein Determining one or more resulting first entity references includes: determining the one or more resulting first entity references based on corresponding distances of the one or more resulting first entity references satisfying the distance criterion.

3. The method according to claim 2, wherein: The distance criterion is an implicit term, and the method further comprises determining the distance criterion based on a distance criterion predefined for the implicit term.

4. The method according to claim 2, further comprising: A user-adjustable graphical element is caused to be displayed in the graphical user interface with which the user can interact to adjust the distance criterion.

5. The method according to claim 4, further comprising: In response to a user interaction with the user adjustable graphical element to widen the distance criterion to a widened distance criterion: At least one additional first candidate entity reference that meets the broadened distance criterion but does not meet the distance criterion is added as an additional annotation on the map of the graphical user interface.

6. The method according to claim 4, further comprising: In response to a user interaction with the user adjustable graphical element to reduce the distance criterion to a reduced distance criterion: At least one of the one or more result first entity references that does not satisfy the reduced distance criteria is caused to be removed from a map of the graphical user interface.

7. The method according to claim 1, wherein: The data structure includes a knowledge graph.

8. A method implemented by one or more processors, the method comprising: Determine that the received combined query includes: The first entity type, The second entity reference, and Distance standards; Identify from the data structure: a plurality of first candidate entity references of the first entity type, and each corresponding position of the first candidate entity reference and the second entity reference; comparing the corresponding position of each of the first candidate entity references with the corresponding position of the second entity reference to determine a corresponding distance between each of the first candidate entity references and the second entity reference; determining the one or more resulting first entity references from the first candidate entity references based on corresponding distances of the one or more resulting first entity references satisfying the distance criterion, and In response to the combined query, the following is displayed as a label on a map of a graphical user interface: the second entity references, and At least one of the one or more result first entity references.

9. The method according to claim 8, wherein: The distance criterion is implicit, And it also includes determining the distance standard based on a distance standard predefined for the implicit term.

10. The method according to claim 9, further comprising: A user-adjustable graphical element is caused to be displayed in the graphical user interface with which the user can interact to adjust the distance criterion.

11. The method according to claim 10, further comprising: In response to a user interaction with the user adjustable graphical element to widen the distance criterion to a widened distance criterion: At least one additional first candidate entity reference that meets the broadened distance criterion but does not meet the distance criterion is added as an additional annotation on the map of the graphical user interface.

12. The method according to claim 10, further comprising: In response to a user interaction with the user adjustable graphical element to reduce the distance criterion to a reduced distance criterion: At least one of the one or more result first entity references that does not satisfy the reduced distance criteria is caused to be removed from a map of the graphical user interface.

13. The method according to claim 8, further comprising: A user-adjustable graphical element is caused to be displayed in the graphical user interface with which the user can interact to adjust the distance criterion.

14. The method according to claim 13, further comprising: In response to a user interaction with the user adjustable graphical element to widen the distance criterion to a widened distance criterion: At least one additional first candidate entity reference that meets the broadened distance criterion but does not meet the distance criterion is added as an additional annotation on the map of the graphical user interface.

15. The method according to claim 13, further comprising: In response to a user interaction with the user adjustable graphical element to reduce the distance criterion to a reduced distance criterion: At least one of the one or more result first entity references that does not satisfy the reduced distance criteria is caused to be removed from a map of the graphical user interface.

16. A method implemented by one or more processors, the method comprising: receiving user input indicating a first entity type, at least one second entity reference, and a distance relationship between a plurality of first entity references of the first entity type and the at least one second entity reference, wherein the distance relationship defines a distance criterion; determining from a data structure a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion; as well as In response to determining a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion: To display it as a callout on the GUI map: the second entity references, and A plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion.

17. The method according to claim 16, wherein: The distance criterion is determined based on a user interaction with a graphical element on the display screen.

18. The method according to claim 17, further comprising: In response to a user interaction with the graphical element to widen the distance criterion to a widened distance criterion: At least one additional entity reference of the first type that meets the broadened distance criteria but does not meet the distance criteria is added as an additional annotation on the map of the graphical user interface.

19. The method according to claim 18, further comprising: In response to a user interaction with the graphical element to reduce the distance criterion to a reduced distance criterion: At least one of the plurality of entity references of the first entity type that does not meet the reduced distance criteria is caused to be removed from a map of the graphical user interface.

20. A system for providing search results, comprising: database; as well as One or more processors configured to perform operations comprising: Determine based on combined query: The first entity type, The second entity type, and Distance standards; Identify from the database: a plurality of first candidate entity references of the first entity type, and a second entity reference of the second entity type; Determining from the database an attribute value including a corresponding position of each of the plurality of first candidate entity references and the second entity reference; comparing the attribute value of each of the first candidate entity references with the attribute value of the second entity reference to determine a corresponding distance between each of the first candidate entity references and the second entity reference; determining one or more result first entity references from the first candidate entity references based on the corresponding distance and the distance criterion, and In response to receiving the combined query and displaying it as a label on a map of a graphical user interface: the second entity references, and At least one of the one or more result first entity references.

21. A system for providing search results, the system comprising: database; as well as One or more processors configured to perform operations comprising: Determine that the received combined query includes: The first entity type, The second entity reference, and Distance standards; Identify from the database: a plurality of first candidate entity references of the first entity type, and each corresponding position of the first candidate entity reference and the second entity reference; comparing the corresponding position of each of the first candidate entity references with the corresponding position of the second entity reference to determine a corresponding distance between each of the first entity references and the second entity reference; determining the one or more resulting first entity references from the first candidate entity references based on corresponding distances of the one or more resulting first entity references satisfying the distance criterion, and To display it as a callout on the GUI map: the second entity references, and At least one of the one or more result first entity references.

22. A system for providing search results, the system comprising: database; as well as One or more processors configured to perform operations comprising: receiving user input indicating a first entity type, at least one second entity reference, and a distance relationship between a plurality of first entity references of the first entity type and the at least one second entity reference, wherein the distance relationship defines a distance criterion; determining from the database a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion; and In response to determining a plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion, To display it as a callout on the GUI map: the second entity references, and A plurality of entity references of the first entity type, the plurality of entity references having corresponding positions relative to a position of the second entity reference that satisfy the distance criterion.

Images