Geographic language model processing method and device, program product and storage medium

Abstract

The embodiment of the invention provides a processing method and device of a geographic language model, a program product and a storage medium. The method comprises the steps that geographic information of the following three types of geographic entities in a preset geographic range is obtained, wherein the three types of geographic entities include a point of interest (POI), an area (AOI) and a road Road; constructing data representation of each POI according to geographic information of each geographic entity; wherein the data representation of the POI comprises an address description text, longitudes and latitudes, and association information for describing an AOI and / or a Road having an association relationship with the POI; the association relationship comprises an inclusion relationship representing that the AOI contains the POI and a proximity relationship representing that the AOI is close to the POI or the Road is close to the POI; and training a preset geographic language model based on the data representation of each POI.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of geographic information technology, and in particular to a processing method, device, program product, and storage medium for a geographic language model. Background Art

[0002] At present, many service scenarios involve the processing of geographic information. Taking the instant delivery scenario as an example, it involves processes such as user address construction, merchant address construction, and rider-based address delivery. In related technologies, the training data used by the geographic language model does not have sufficient understanding of the address, resulting in poor model performance. Summary of the invention

[0003] To overcome the problems existing in the related art, the embodiments of this specification provide a method, device, program product and storage medium for processing a geographic language model.

[0004] According to a first aspect of an embodiment of this specification, a method for processing a geographic language model is provided, the method comprising:

[0005] Get the geographic information of the following three types of geographic entities within the preset geographic range: point of interest POI, area AOI and road Road;

[0006] According to the geographic information of each geographic entity, construct a data representation of each POI; wherein the data representation of the POI includes: address description text, longitude and latitude, and association information describing the AOI and / or Road associated with the POI; the association relationship includes: an inclusion relationship representing that the AOI includes the POI, and an adjacency relationship representing that the AOI is adjacent to the POI or the Road is adjacent to the POI;

[0007] Based on the data representation of each of the POIs, a preset geographic language model is trained.

[0008] According to a second aspect of an embodiment of this specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method embodiment described in the first aspect are implemented.

[0009] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method embodiment described in the first aspect are implemented.

[0010] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method embodiment described in the first aspect are implemented.

[0011] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0012] In the embodiments of this specification, a data representation of POI is constructed based on geographic information of three types of geographic entities, namely, points of interest (POI), areas (AOI), and roads (Road); wherein two types of association relationships are designed, namely, an inclusion relationship that represents that AOI includes POI, and an proximity relationship that represents that AOI is adjacent to POI or Road is adjacent to POI. In addition, the information of AOI and / or Road associated with POI is designed to be included in the data representation of POI. Therefore, the data representation of a POI not only includes the information of the POI itself, but also includes the information of AOI and / or Road at the location and surrounding areas of the POI. Therefore, the data representation of POI in this embodiment can accurately represent the spatial relationship between geographic entities, achieve accurate description of spatial relationships, and enable the geographic language model trained based on the data representation of POI in this embodiment to accurately learn the information of POI itself, as well as the information of AOI and / or Road at the location and surrounding areas of the POI.

[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1A It is a flowchart of a method for processing a geographic language model according to an exemplary embodiment of the present specification.

[0015] Figure 1B It is a schematic diagram of an association relationship between geographic entities according to an exemplary embodiment of the present specification.

[0016] Figure 1C It is a schematic diagram for explaining data representation of POIs in a table according to an exemplary embodiment of the present specification.

[0017] Figure 1D It is a schematic diagram of an AOI in a geographic grid according to an exemplary embodiment of this specification.

[0018] Figure 2A It is a schematic diagram of a geographical comparison learning task according to an exemplary embodiment of this specification.

[0019] Figure 2B It is a schematic diagram of a user interface shown in this specification according to an exemplary embodiment.

[0020] Figure 2C It is another user interface schematic diagram shown in this specification according to an exemplary embodiment.

[0021] Figure 2D It is another user interface schematic diagram shown in this specification according to an exemplary embodiment.

[0022] Figure 3 It is a hardware structure diagram of a computer device where a processing device of a geographic language model is located according to an exemplary embodiment of this specification.

[0023] Figure 4 It is a block diagram of a processing device for a geographic language model according to an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0025] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] At present, many service scenarios involve the processing of geographic information. Taking the instant delivery scenario as an example, it involves processes such as user address construction, merchant address construction, and rider-based address delivery. When a user places an order, the address needs to be constructed. The user can search by keyword in the user client. The user client will provide multiple recommended POIs to the user based on the keyword. The user can click on a POI. After the user client determines the POI selected by the user, the user can further complete the house number, phone number or contact person to complete the user address construction process. If the user does not know the name of the POI, or wants to view the recommended POI, the user can use the surrounding search function of the user client. The client will obtain the current geographic location and obtain multiple nearby recommended POIs for the user, so that the user can continue to complete the user address construction process. In vertical fields such as takeout, address-related information, in addition to text information, also has geographic information, such as: Bird's Nest, longitude and latitude (39.993306, 116.395866).

[0029] At present, some open source pre-trained language models are pre-trained on Internet public book corpora and encyclopedia corpora, which can greatly improve the performance of text-related tasks, such as text embedding. For example, the keyword search mentioned above involves the calculation of the similarity of address text and the embedding of address text, so the pre-trained language model can be used to fine-tune the address text and calculate the similarity of address text. However, the pre-trained language model has two disadvantages. One is that the pre-trained corpus of the model is a general corpus, and the address text in scenarios such as takeout is a vertical corpus, and the scenario is not an exact match; the other is that the surrounding search function hopes to describe the coordinate information of longitude and latitude, but the pre-trained language model can only describe text information, which makes the surrounding search function unusable.

[0030] Based on this, a geographic language model can be designed to introduce geographic context as a new modality on the text information of a single modality. A dual encoder architecture can be used in the model, including a text encoder for processing text and a geo encoder for processing geographic representation, so that the multimodal geographic language model can be improved in its ability to understand and process geographic information. The main goal of the model is to learn the association between a geographic entity, such as the text attributes of a POI (such as the name of the POI "Beijing West Railway Station" and the address of the POI "No. 118 Lianhuachi East Road, Fengtai District, Beijing") and its corresponding geographic coordinates (the latitude and longitude of the POI).

[0031] However, the current processing of geographic language models still has defects; for example, there are defects in the design of the model's training data. Some solutions are limited to the longitude and latitude of the geographic entity itself and the distance between the geographic entities, and the information such as spatial orientation relationship is lost; for large geographic entities, such as communities, the buildings in the community may be far apart, but for community management, they are actually located in one area (a certain community); therefore, the distance cannot be used alone to characterize the relationship between buildings. In other solutions, the roads in the geographic entity are represented by a single longitude and latitude, which also leads to inaccurate characterization of the relationship information between POI and roads. Some solutions use open source map datasets, such as OSM (Open StreetMap) data. These data sets do not adequately characterize the address granularity of the community, building, house number, merchant address, etc. in the takeaway scenario.

[0032] Based on this, the embodiment of this specification provides a method for processing a geographic language model, and constructs a data representation of POI based on the geographic information of three types of geographic entities, namely, points of interest (POI), areas (AOI), and roads (Road); wherein, two types of association relationships are designed, namely, the inclusion relationship that represents that AOI includes POI, and the proximity relationship that represents that AOI is adjacent to POI or Road is adjacent to POI. In addition, the information of AOI and / or Road associated with POI is designed to be included in the data representation of POI, so that the data representation of a POI not only includes the information of the POI itself, but also includes the information of AOI and / or Road at the location and surrounding areas of the POI. Therefore, the data representation of POI in this embodiment can accurately represent the spatial relationship between geographic entities, realize the precise characterization of spatial relationships, and also enable the geographic language model trained based on the data representation of POI in this embodiment to accurately learn the information of POI itself, and also learn the information of AOI and / or Road at the location and surrounding areas of the POI.

[0033] like Figure 1A FIG. 1 is a flowchart of a method for processing a geographic language model according to an exemplary embodiment of the present specification, which may include the following steps:

[0034] In step 102, geographic information of the following three types of geographic entities within a preset geographic range is obtained: point of interest POI, area AOI and road Road.

[0035] In step 104, a data representation of each POI is constructed according to the geographic information of each geographic entity.

[0036] The data representation of the POI includes: address description text, longitude and latitude, and association information describing the AOI and / or Road associated with the POI; the association relationship includes: an inclusion relationship representing that the AOI includes the POI, and an proximity relationship representing that the AOI is adjacent to the POI or the Road is adjacent to the POI.

[0037] In step 106, a preset geographic language model is trained based on the data representation of each of the POIs.

[0038] First, the underlying data of the geographic language model of this embodiment is described. The underlying data involved in this embodiment includes: POI, AOI, road and relation;

[0039] ①POI; a point on a map; it can have a name, address description text, and latitude and longitude;

[0040] For example, the name of a POI is: Building 9, District 2, Anzhen Xili; the address is: 180 meters west of the intersection of Anzhen Xili Ring Road and Anzhen West First Road, Chaoyang District, Beijing; the longitude and latitude are: longitude lng: 116.397348, latitude lat: 39.974515.

[0041] ②AOI; on the map, it is a surface, which can also be considered as a fence; it can have a name, address description text address and latitude and longitude;

[0042] For example, the name of an AOI is: Anzhen Xili District 2; the address is: 180 meters west of the intersection of Anzhen Xili Ring Road and Anzhen West 1st Road, Chaoyang District, Beijing; the longitude and latitude are: lng: 116.396307, lat: 39.973943;

[0043] ③Road; a route on a map; can have a name;

[0044] For example, the name of a road is: Anzhen Xili Ring Road. In addition to the name field, the road consists of multiple sections, and the information of each section is represented by the latitude and longitude of the starting point.

[0045] The design of the relationship between geographic entities can be:

[0046] ① Inclusion relationship, that is, the point is inside the fence, which is called inclusion relationship; there is only one case of inclusion relationship, that is, POI is inside AOI, and AOI contains POI;

[0047] ② Proximity relationship, that is, the point is near the fence, which is called proximity relationship; there are two cases: the POI is not in the AOI, but near the AOI; and the POI is near the Road, which are collectively called proximity relationship. Among them, the proximity measurement standard can be set to set the distance between the POI and the AOI to be less than or equal to the preset distance threshold, and can be set to set the distance between the POI and the Road to be less than or equal to the preset distance threshold. The specific measurement standard can be determined as needed in actual application, and this embodiment does not limit this.

[0048] For example, Figure 1B As shown, it is a schematic diagram of the association relationship between geographic entities according to an exemplary embodiment of the present specification. In the figure, the large ellipse is a certain AOI, the four sides of the square can represent four roads respectively, and the four small ellipses contained in the large ellipse are POIs. It can be seen that the AOI contains 4 POIs, and the POI has a proximity relationship with a certain Road.

[0049] As an example, based on the application scenario of this embodiment, map data in a vertical field can be pre-acquired to construct the data representation of the above-mentioned geographic entities, and then used for processing the geographic language model. As an example, in the case of application to an instant delivery service scenario, existing data in the scenario can be obtained; for example, in the process of providing instant delivery services, the server can obtain the delivery address filled in by the user when placing an order or the merchant address filled in by the merchant with the authorization of the user or the merchant. In addition, other data can also be combined, such as map data of existing map manufacturers or other open source data, etc.

[0050] In some examples, the association information includes one or more of the following:

[0051] The ID of the associated geographic entity, the type of the associated geographic entity, the type of the association relationship between the POI and the associated geographic entity, the absolute position of the associated geographic entity in the geographic grid, and the distance information between the POI and the associated geographic entity;

[0052] The associated geographic entity includes the AOI and / or Road associated with the POI.

[0053] As an example, for the convenience of illustration, a table is used to show the data representation of a POI. It can be understood that the data representation of POI in actual applications may not be in a table format. Each row in the table represents a field, that is, each type of information in the data representation. Each type of information can be surrounded by a specific symbol to distinguish different information. For example, a pair of brackets is used in the table, and a pair of brackets represents a type of information. See the table:

[0054]

[0055] The first 8 rows in the above table represent the association information between the POI and its three associated geographic entities (referred to as geometric entities in the above table for short);

[0056] Please see Figure 1C As shown in FIG. 1 , it is a schematic diagram for explaining the data representation of POI in the table in this embodiment, combining the table and Figure 1C :

[0057] (1) Lines 1 to 4 are used to collectively represent the ID of the geometric body.

[0058] (2) The fifth line is used to indicate the type information of the geometric body, that is, whether the geometric body is AOI or Road. In the table, the number 3 is used to indicate AOI, and the number 4 is used to indicate Road. It can be understood that other numbers or other characters can be used in actual applications to indicate the type information.

[0059] (3) The sixth row is used to indicate the association relationship between the POI and the geometric body, that is, whether it is a containment relationship or a proximity relationship. In the table, the number 3 is used to indicate a containment relationship, and the number 4 is used to indicate a proximity relationship. It is understandable that other numbers or other characters can also be used in actual applications.

[0060] A POI can have one or more associated geographic entities. Figure 3 In the example of 3, in each row of fields, the corresponding information of each associated geographic entity is separated by a specific symbol (comma in the table); for example, Figure 1C Among the three associated geographic entities (①, ② and ③) shown in Figure 1C The information corresponding to each row in the table, that is, the part framed by the dotted box, is: 17, 18, 6, 174, 3, 4. The same is true for associated geographic entities ② and ③. For example, the information corresponding to associated geographic entity ② is: 17, 18, 6, 139, 3, 4.

[0061] (4) The seventh row in the table indicates the absolute position in the geographic grid, that is, the geographic location range of each geometric body; as an example, the absolute position of the associated geographic entity in the geographic grid can be determined in the following manner:

[0062] Divide the preset geographical range into a plurality of rectangular geographical grids according to rows and columns, and configure a row number and a column number for each geographical grid; wherein the row direction is the latitude direction, and the column direction is the longitude direction;

[0063] Obtain the row number and column number of the geographic network where the lower left corner of the associated geographic entity is located, as well as the row number and column number of the geographic network where the upper right corner of the associated geographic entity is located, and use the obtained numbers as the absolute position of the associated geographic entity in the geographic grid; wherein the lower left corner refers to the southwesternmost point of the associated geographic entity on the map, and the upper right corner refers to the northeasternmost point of the associated geographic entity on the map.

[0064] For example, Figure 1D , which is a schematic diagram of a certain AOI in a geographic grid shown in an embodiment of this specification; in the figure, taking the geographic field as an example, the top is north, the bottom is south, the left is west, and the right is east, O1 is the point at the lower left corner of the AOI, and O2 is the point at the upper right corner of the AOI;

[0065] In the geographic coordinate system, the left side corresponds to smaller longitude (west), and the right side corresponds to larger longitude (east); the upper side corresponds to larger latitude (north), and the lower side corresponds to smaller latitude (south).

[0066] The entire map area within a preset geographic range (such as a custom range such as a country) can be divided into multiple geographic grids, and the shape and size of the geographic grid can be customized. As an example, suppose it is divided into N×M geographic grids, which are divided into N rows (latitude direction) and M columns (longitude direction). The row number and column number can be configured for each geographic grid unit; for example:

[0067] Row number (row number): numbered from top to bottom (north to south), for example, the first row is 1, the second row is 2, and so on, to N rows.

[0068] Column number (column number): numbered from left to right (west to east), for example, the first column is 1, the second column is 2, and so on, to column M.

[0069] Thus, the aforementioned geographical location range can be represented based on the number of the geographical grid where it is located. For example, the row and column numbers where the upper left corner and the lower right corner of the geometric body are located are combined into four numbers to represent the coverage of a geographical entity (such as AOI or Road). For example, a similar format such as [lower left row number, lower left column number, upper right row number, upper right column number] can be used.

[0070] For example, the first pair of brackets "[418,1391,418,1392]" represents the geographical location range of the associated geographical entity ①, and the geographical location range is represented by the position of the lower left corner and the position of the upper right corner. In actual applications, the lower left corner and the upper right corner have longitude and latitude, which can be represented by longitude and latitude. In this embodiment, a discrete 4-digit number is used; of course, the specific data length in actual applications can be set as needed.

[0071] For example, in geographic information processing, longitude and latitude coordinates are continuous and represent any point on the earth's surface. However, directly using continuous coordinates will result in a large amount of data and high computational complexity. Discretization makes the data more concise and easier to manage by dividing the continuous space into a finite number of grid cells and mapping each point to a specific grid number.

[0072] For example, the geographic location range [418,1391,418,1392] of the associated geographic entity ① indicates that the lower left corner of the geographic entity is located in the geographic grid "418,1391" and the upper right corner is located in the geographic grid "418,1392".

[0073] Based on this, continuous latitude and longitude coordinates can be converted into a finite number of discrete grid cells, which can accurately represent the absolute positions of associated geographic entities in the map network for easy model processing and calculation.

[0074] (5) The 8th row in the table represents the distance information between the POI and the associated geographic entity, that is, the distance between the longitude and latitude of the POI and each geometric entity;

[0075] As an example, the distance information between the POI and the associated geographic entity may be determined in the following manner:

[0076] A first distance between the POI and the lower left corner of the associated geographic entity and a second distance between the POI and the upper right corner of the associated geographic entity are calculated, and distance information between the POI and the associated geographic entity is determined based on the first distance and the second distance.

[0077] As an example, the first distance and the second distance here may be absolute distances, which may be obtained by coordinate calculation.

[0078] Optionally, in other examples, the first distance may include: the longitude distance between the POI and the lower left corner of the associated geographic entity, and the latitude distance between the POI and the lower left corner of the associated geographic entity; the second distance may include: the longitude distance between the POI and the upper right corner of the associated geographic entity, and the latitude distance between the POI and the upper right corner of the associated geographic entity.

[0079] The first distance and the second distance may be discretized to obtain the distance information between the POI and the associated geographic entity. For example, the discretization of the distance in this embodiment may refer to dividing the continuous distance value (such as the real distance in the longitude or latitude direction) into a plurality of predefined distance intervals, and assigning a corresponding number or label to each distance interval. In this way, the originally infinite number of continuous values ​​is simplified into a limited and manageable category.

[0080] For example, the numbers from 1 to 50 represent 50 different distance intervals respectively. The size of each distance interval can be set according to actual needs, which is not limited in this embodiment.

[0081] For example, in the table "[30,38,21,38]", where:

[0082] 30, 38: together represent the first distance between the POI and the lower left corner of the associated geographic entity ①, including:

[0083] 30 is the discretized value of the longitude distance between the POI and the lower left corner. In other words, 30 represents the number of the distance interval in which the longitude distance between the POI and the lower left corner is located.

[0084] 38 is the discretized value of the latitude distance between the POI and the lower left corner. Similarly, 38 represents the number of the distance interval in which the longitude distance between the POI and the lower left corner is located.

[0085] 21, 38: jointly represent the second distance between the POI and the upper right corner of the associated geographic entity ①, including:

[0086] 21, which is the discretized value of the longitude distance between the POI and the upper right corner.

[0087] 38, which is the discretized value of the latitude distance between the POI and the upper right corner.

[0088] (6) The 9th row in the table represents the address description text of the POI, that is, the text describing the geographical location of the POI.

[0089] For example, the address description text includes the POI name and the POI address, and the POI name is separated by a specific symbol (such as ∮) between the POI addresses. For example, "Building 18, Jinxiu Garden" is the name of the POI, and "Jinxiu Garden, Hejing County, Bayinguoleng Mongolian Autonomous Prefecture" is the address of the POI.

[0090] (8) The 10th row in the table indicates the number of the POI in the data set, that is, the index idx in the data set. The "0" in the example here can indicate the first data item in the data set. In actual applications, it is optional to not include the index field as needed.

[0091] (9) Row 11 in the table indicates the longitude and latitude of the POI.

[0092] In actual applications, if the preset geographic range is large, the number of AOI and Road IDs will be huge. For example, if each geometric body is configured with an ID starting from 0, there will be millions of AOIs, and the ID value will range from 1 to millions. In order to reduce the number of IDs, the IDs of each AOI and / or Road within the preset geographic range can be configured as follows:

[0093] Dividing the preset geographical scope into multiple administrative levels, and configuring corresponding administrative unit IDs for each administrative unit in each administrative level;

[0094] Numbering each associated geographic entity contained in the lowest administrative level in the multiple administrative levels;

[0095] The administrative unit ID of each administrative level to which the associated geographic entity belongs and the number are combined to obtain the ID of the associated geographic entity.

[0096] In order to optimize the management of geometric IDs, this embodiment adopts a hierarchical design according to administrative levels, so that the geometric IDs in each administrative unit are counted starting from 1. This method can significantly reduce the number of globally unique IDs.

[0097] For example, the default geographical scope is the country, and the multi-level administrative hierarchy is 3 layers, including provinces, cities, and districts;

[0098] Provincial ID (pname_id): Each province is assigned a unique provincial ID, for example, from 1 to 32.

[0099] City ID (cname_id): Each city is assigned a unique city ID within its province, for example, from 1 to 18.

[0100] District ID (adname_id): Each district is assigned a unique district ID within the city to which it belongs, for example, from 1 to 08. Quickly locate and manage geometry based on administrative levels.

[0101] Geometry ID Number:

[0102] Geometry ID (geom_id): In each zone, the geometry (AOI or Road) is numbered starting from 1, for example 174, 139, 021, etc.

[0103] The ID of each geometric body can be obtained by combining the ID at the province, city, and district levels with the geometric body number.

[0104] As an example, to ensure uniqueness and consistency, a fixed-length encoding method can be used. For example:

[0105] Provincial ID: 2 digits

[0106] City ID: 2 digits

[0107] District ID: 2 digits

[0108] Geometry number: 4 digits

[0109] The IDs of the three geometries in the table can be:

[0110] Geometry 1:

[0111] pname_id:17+cname_id:18+adname_id:06+geom_id:0174→[17,18,06,0174]

[0112] Geometry 2:

[0113] pname_id:17+cname_id:18+adname_id:06+geom_id:0139→[17,18,06,0139]

[0114] Geometry 3:

[0115] pname_id:17+cname_id:18+adname_id:06+geom_id:0021→[17,18,06,0021]

[0116] Assume that there are 4 million geometric bodies. If a globally unique ID is adopted, i.e., all geometric bodies are assigned a unique ID, which needs to be from 0 to 3999999. This method needs to process and store a large number of unique IDs, resulting in a sharp increase in memory usage. The hierarchical design of the present embodiment will be hierarchically managed according to administrative divisions such as provinces, cities, and districts, and the geometric body IDs in each level can be renumbered. The geometric body IDs in each province, city, and district are all locally unique, rather than globally unique. By combining hierarchical information (such as province ID, city ID, district ID) and local ID, each geometric body can be uniquely identified. A complete geometric body identifier can be composed of a hierarchical identifier (such as province ID+city ID+district ID) and a local ID. For example, the complete ID of a geometric body can be expressed as [province ID, city ID, district ID, local ID], rather than a single global ID.

[0117] Based on the data representation of the geographic entity constructed above, the preset geographic language model can be trained. The geographic language model of this embodiment can include a text encoder and a geocoder, the text encoder is used to encode the address description text address_text in the data representation, and the geocoder is used to encode the geographic representation in the data representation, and the geographic representation is the information in the data representation other than the address description text. For example, in the above table, the other information except address_text is the geographic representation here.

[0118] Optionally, the model structure of the geographic language model of this embodiment can be constructed by itself, or an existing geographic language model can be adopted, or an existing geographic language model can be improved on the basis of the existing geographic language model.

[0119] Optionally, the text encoder can be configured according to actual needs. For example, you can choose a 12-layer BERT (Bidirectional Encoder Representation from Transformers) model, which has more layers and is suitable for offline scenarios. You can also choose a 4-layer tiny BERT model, which is suitable for online scenarios that consider timeliness.

[0120] In this embodiment, the geographic language model is processed using a two-stage training method, as an example:

[0121] Constructing an initial geographic language model with initial parameters; the initial geographic language model includes a pre-trained text encoder and a geographic encoder;

[0122] Performing a training task related to the geographic representation on the initial geographic language model to obtain a pre-trained geographic language model;

[0123] After fixing the parameters of the geocoder in the pre-trained geo-language model, the pre-trained geo-language model is subjected to a training task related to the address description text to obtain a pre-trained geo-language model.

[0124] For example, the aforementioned constructed data can be used to pre-train the training tasks related to geographic representation in the first stage; wherein the training tasks related to geographic representation can include geographic mask modeling tasks or geographic contrast learning tasks, etc.; in the first stage of training, a loss function can be designed to complete the pre-training of the geocoder. The geocoder can start with no pre-trained parameters (random initialization), and be trained through the designed loss function to gradually optimize its parameters.

[0125] After completing the first phase of training, in the second phase, geography and text can be integrated, the parameters of the geocoder can be fixed, and the focus can be on learning text features. Unlike the geocoder, the text encoder can use pre-trained language models (such as the Bert model) as a basis. These models are pre-trained on large-scale corpora and already have rich semantic understanding capabilities.

[0126] Next, the training tasks involved in the two-stage training process are described. In practical applications, one or more of the following training tasks may be used, which are not limited in this embodiment.

[0127] (1) Mask Geographic Modeling (MGM);

[0128] As an example, the training tasks related to the geographic representation include the following: geographic mask modeling task:

[0129] A portion of the geographic representation in the data representation of the POI is masked, and the masked data representation of the POI is input into the geographic language model so that the geographic language model predicts prediction information of the masked position, and is trained based on an error between the prediction information and the masked real information in the data representation of the POI.

[0130] As an example, any of the features pname_id, cname_id, adname_id, geom_id, geom_type_id, geom_rel_type_id, gis_absolute_position_ids, and gis_relative_position_ids can be masked, and the model can calculate the loss based on the difference between the predicted value and the true value for the value at the predicted mask position.

[0131] (2) Geographic Contrastive Learning (GCL);

[0132] As an example, the training task related to the geographic representation includes the following geographic comparison learning task:

[0133] The data representations of any two POIs are obtained and input into the geographic language model, so that the geographic language model calculates the vector similarity of the two POIs based on the geographic representations in the data representations of the two POIs, and is trained based on the geographic distance between the longitude and latitude in the data representations of the two POIs to minimize the error between the vector similarity and the geographic distance.

[0134] like Figure 2A , which is a schematic diagram of a geographical comparison learning task according to an exemplary embodiment of the present specification; Figure 2A The data includes two POIs A and B. The distance between map location A and map location B can be calculated based on the longitude and latitude in the data representation.

[0135] The geo representation of A is the geo representation in the data representation of A, and the geo representation of B is the geo representation in the data representation of B. The vector similarity between the geo representation of A and the geo representation of B is measured by L2 similarity in this embodiment. L2 similarity is the Euclidean distance, which can be obtained by calculating the square root of the sum of the squares of the differences between the two points in each dimension. In practical applications, other similarity measurement indicators such as cosine distance can also be used, which is not limited in this embodiment.

[0136] In the geographic contrastive learning task, the training goal of the model is to ensure that the similarity (such as L2 distance) of the geographic representation vector is consistent with the actual geographic distance between POIs. Specifically, geographically close POIs should have higher similarity (smaller L2 distance) in the vector space, while geographically far POIs should have lower similarity (larger L2 distance).

[0137] Based on this, a loss function can be designed so that the vector similarity learned by the model can reflect the actual geographical distance. The specific loss function can be designed according to actual needs.

[0138] (3) MGM&GCL, which is the task that integrates the two training tasks mentioned above;

[0139] The training tasks related to the geographic representation include geographic mask modeling and geographic contrast learning joint tasks:

[0140] Get the real data representation of any two POIs;

[0141] Masking parts of the geographic representations in the data representations of the two POIs respectively, and inputting the masked data representations of the two POIs into the geographic language model, so that the geographic language model:

[0142] Predicting the prediction information of the mask position in the mask data representation of the two POIs, after obtaining the predicted data representation of the two POIs, calculating the vector similarity of the two POIs based on the geographic representation in the predicted data representation of the two POIs, and then training based on the geographic distance between the longitude and latitude in the real data representation of the two POIs to minimize the error between the vector similarity and the geographic distance; wherein the geographic representation is other information in the data representation except the address description text.

[0143] As an example, the distance between the longitude and latitude of points A and B can be calculated, and the geo representation of A and the geo representation of B can be masked and predicted and restored. Finally, the learned distance and the L2 similarity of the geo representation can be compared for comparative learning.

[0144] (4) Mask Language Modeling (MLM)

[0145] As an example, the training task related to the address description text includes the following language mask modeling task:

[0146] Part of the characters of the geographic description text in the data representation of the POI is masked, and the masked data representation of the POI is input into the geographic language model so that the geographic language model predicts the predicted characters at the masked positions, and is trained based on the errors between the predicted characters and the real characters masked in the data representation of the POI.

[0147] For example, any character in address_text can be masked, the model can predict the masked character, and the error between the predicted character and the real character in the data representation can be used for training.

[0148] In some examples, the geographic description text includes: a POI name and a POI address; the masked information in the address description text may include the POI name or the POI address. That is, a sentence pair similarity pre-training task may be performed.

[0149] Please see the address_text field in the aforementioned table, "Building 18, Jinxiu Garden ∮ Jinxiu Garden, Hejing County, Bayinguoleng Mongolian Autonomous Prefecture", the content before the symbol ∮ is the name of the POI, and the content after the symbol ∮ is the address of the POI. This embodiment actually concatenates the name and address using specific symbols, so that the field is actually a sentence pair.

[0150] Therefore, this embodiment can perform a sentence pair similarity training task, and the sentence pair similarity training task can be expressed as:

[0151] X inp =[CLS]S1[SEP]S2[SEP]

[0152] Among them, X inp is the target function, S1 represents the name (POI name) in the address field address_text, S2 represents the address (physical address name of the POI) in the address field address_text, and [CLS] and [SEP] are separators.

[0153] In this way, this embodiment can train the model to learn the same referential expression of a sentence pair.

[0154] (5)MGM&MLM (Multi-Modal MLM);

[0155] The training tasks related to the address description text include the following joint tasks of geographic mask modeling and language mask modeling:

[0156] Masking part of the address description text in the real data representation of the POI, and masking part of the geographical representation in the data representation of the POI, and inputting the masked data representation of the POI into a preset geographic language model, so that the geographic language model:

[0157] The information of the masked address description text in the masked data representation is predicted, and the information of the masked geographic representation in the masked data representation is predicted. After obtaining the predicted data representation of the POI, training is performed to minimize the error between the predicted data representation of the POI and the true data representation.

[0158] That is, the text of the address can be masked and predicted and restored, and the geo representation of the address can be masked and predicted and restored at the same time, the text vector and the geo representation vector can be concatenated, and then the Loss can be calculated.

[0159] In some examples, an adversarial attention mechanism can also be introduced. Adversarial training improves the robustness of the model by adding perturbations to the input content. As an example, the text encoder can be connected to a first adversarial attention module, which is used to generate a first interference coefficient so that the text encoder encodes the address description text in the data representation based on the first interference system; and / or, the geocoder is connected to a second adversarial attention module, which is used to generate a second interference coefficient so that the geocoder encodes the geographic representation based on the second interference system.

[0160] As an example, the text encoder can be an encoder based on Transform, which can include a multi-layer Transform structure, and the specific number of layers is not limited in this embodiment. Each layer in each Transformer is composed of a self-attention mechanism and a feedforward neural network, and the performance of the model is improved through residual connections and layer normalization. Similarly, the geocoder can also be an encoder based on Transform, which can include a multi-layer Transform structure, and the specific number of layers is not limited in this embodiment.

[0161] As an example, adversarial perturbations can be introduced into the self-attention mechanism in a specific layer of the encoder. For example, each layer in the multi-layer Transform structure of the text encoder can be selected, or some of the layers can be selected as specific layers here. The specific layers selected to introduce adversarial perturbations can be set according to actual needs, thereby suppressing the model's dependence on specific keywords and prompting the model to explore a wider range of semantic information, thereby improving the generalization ability and robustness of the model. Similarly, specific layers can be selected in the multi-layer Transform structure of the geocoder to introduce adversarial perturbations.

[0162] As an example, the adversarial attention module can generate an interference coefficient. For example, the interference coefficient can specifically be an adversarial mask matrix introduced in the self-attention layer, which can be generated by learning to adjust the distribution of attention scores.

[0163] Optionally, a variety of training tasks are designed in the aforementioned embodiment, and the adversarial attention mechanism of this embodiment may be introduced in all training tasks or in one or more of the training tasks, which is not limited in this embodiment.

[0164] In practical applications, there are multiple processing tasks for the geographic language model. If the method of executing each training task in series is adopted, the model may have a catastrophic forgetting problem, that is, after the last training task is completed, the model forgets the knowledge learned in the first training task. In some examples, this embodiment can adopt a non-serial method. For example, in the process of performing multiple training tasks on the geographic language model, the target training tasks are first uniformly sampled from the multiple training tasks to train the geographic language model. After the geographic language model has been trained on all training tasks, the target training tasks are sampled from the multiple training tasks based on the loss value of the loss function of the training task to train the geographic language model; wherein the loss value is positively correlated with the probability of being sampled.

[0165] This embodiment can adopt the method of executing multiple training tasks in parallel. In the initial stage of training, each training task can be evenly sampled, and each training task has an equal probability of being selected for training at the beginning. This uniform sampling strategy ensures that each target can be explored fairly in the absence of prior knowledge. Each time a target training task is sampled from multiple training tasks, the model is trained based on the selected target training task, thereby accumulating preliminary training data and performance indicators. After the model is trained on all training tasks, the target training task can be sampled from the multiple training tasks based on the loss value of the loss function of the training task to train the geographic language model. For example, each time sampling is performed, the training task with a larger loss can be given priority, so as to gradually reduce the loss of each training task, which is compatible with each other and promotes each other.

[0166] Based on the geographic language model trained in this embodiment, a variety of related applications can be performed. Some examples are given below to illustrate this.

[0167] As an example, after the step of training the preset geographic language model, the method may further include:

[0168] The POI vectors generated by the geographic language model for the data representation of each POI are obtained and stored in a vector database.

[0169] In this embodiment, after the geographic language model is trained, the conversion relationship between the data representation of POI and the vector can be learned. Therefore, after the training of the geographic language model is completed, the data representation of each POI can be input into the geographic language model, and the POI vector generated by the geographic language model for the input POI data representation can be obtained, and the POI vector can be stored in the vector database. Since this embodiment uses the trained geographic language model to generate accurate POI vectors and stores them in the vector database, a variety of POI-related services can be provided based on the vector database.

[0170] As an example, the POI vectors stored in the vector database can be used in other tasks of the service provider, such as the delivery time estimation task. In the field of instant delivery, the service provider can be configured with a delivery time estimation model. The model needs to estimate the delivery time for the delivery capacity to deliver the items to the user's delivery address after the user places an order. The input of the delivery time estimation model includes the POI of the current location of the delivery capacity, the POI of the merchant address where the order is placed, and the POI of the delivery address. The vectors of these three POIs can be obtained by the geographic language model of this embodiment. For example, the POI vectors of these three POIs can be queried from the vector database.

[0171] As an example, the vector database may be applied to a query scenario, and the method may further include:

[0172] Receive query information related to POI;

[0173] Inputting the query information into the geographic language model to obtain a query vector of the query information output by the geographic language model;

[0174] A POI vector similar to the query vector is retrieved from the vector database.

[0175] As an example, this embodiment can be applied to query scenarios related to POIs. There are many sources of query information related to POIs, such as clients or other services, which are not limited in this embodiment. The query information can include any information related to POIs, and the geographic language model can generate a corresponding query vector for the query information. After that, POI vectors similar to the query vector can be retrieved from the vector database. Among them, whether the vectors are similar can be achieved by calculating the vector similarity, and the similarity can be measured by cosine similarity or Euclidean distance, etc., and the condition for measuring whether the vectors match can be a condition such as the similarity of the two vectors is greater than or equal to a preset similarity threshold. In this way, this embodiment can use the geographic language model to implement query services related to POIs.

[0176] In some examples, the query information may include: the user's current geographic location information; or the current search term provided by the user.

[0177] Taking the query information as the user's current geographical location information as an example, this embodiment can use the geographical language model to implement the surrounding search function. Figure 2B As shown in FIG. 1 , a user interface diagram is shown in this specification according to an exemplary embodiment. In the user client, the user needs to confirm the delivery address when placing an order. The client can obtain the user's current location (the client can obtain the current longitude and latitude), and the obtained longitude and latitude can be input into the geographic language model. The geographic language model outputs the query vector of the longitude and latitude, and then retrieves a batch of similar vectors matching the query vector from the vector database. The batch of similar vectors can be sorted according to the set strategy and then output to the user. Figure 2B 3 POIs are shown in order below the map.

[0178] Taking the current search term provided by the user in the query information as an example, this embodiment can use the geographic language model to implement the keyword search function. Figure 2CAs shown, it is another user interface schematic diagram shown in this specification according to an exemplary embodiment. In the user client, the user needs to confirm the delivery address when placing an order. The client can enter the current search term in the search bar, and the current search term obtained can be input into the geographic language model. The geographic language model outputs the query vector of the search term, and then retrieves a batch of similar POI vectors similar to the query vector from the vector database. This batch of similar vectors can be sorted according to the set strategy and then output to the user, for example, sorted from high to low according to the degree of similarity. Figure 2C As shown in FIG. 1 , after the user searches, four sorted POIs are displayed under the map.

[0179] Optionally, in the above embodiment, the similarity of the vectors is measured by calculating the cosine similarity or Euclidean distance of the vectors. In this embodiment, for a batch of similar POI vectors retrieved from the vector database, similarity scores may be calculated again to improve the recommendation accuracy.

[0180] The query information includes a current search term provided by a user; after the step of retrieving a vector similar to the query vector from the vector database, the method may further include:

[0181] Splicing the query vector with the retrieved POI vector to obtain a spliced ​​vector and inputting the spliced ​​vector into a similarity score evaluation model, so that the similarity score evaluation model outputs a similarity score between the query vector and the retrieved POI vector;

[0182] Among them, the similarity score evaluation model is obtained by training the geographic language model using splicing vector samples, and the splicing vector samples include: a vector spliced ​​by a historical query vector and a historical click POI vector, and a vector spliced ​​by a historical query vector and a historical non-clicked POI vector; the historical query vector refers to a vector corresponding to a user's historical search terms, the historical click POI vector refers to a vector corresponding to a historical click POI clicked by a user after searching for the historical search terms, and the historical non-clicked POI vector refers to a vector corresponding to a click POI that was not clicked by a user after searching for the historical search terms.

[0183] As an example, the pre-trained geographic language model can be fine-tuned for the similarity score evaluation task. The fine-tuning data set can be constructed based on historical user search behavior and click logs, and the training data constructed from the logs includes: historical search terms, multiple recommended POIs corresponding to the historical search terms, wherein the multiple recommended POIs include historical clicked POIs and historical unclicked POIs clicked by users, and the clicked POI is the POI that is most similar to the historical search term, and the similarity score of the two can correspond to a preset score to indicate that the two are highly similar; each POI that has not been clicked in the history of the multiple recommended POIs is a POI that is not similar to the historical search term, and the similarity score of the two can also be set to a preset score to indicate that the two are not similar. The constructed training data can be used to fine-tune the geographic language model for the similarity score evaluation task. For example, the positive sample used for training is a vector obtained by concatenating the historical query vector and the historical clicked POI vector. After the positive sample is input into the model, the model can predict the predicted similarity score of the two, and adjust the model training parameters according to the difference between the predicted similarity score and the actual similarity score; the negative sample used for training is a vector obtained by concatenating the historical query vector and the historical unclicked POI vector. After the positive sample is input into the model, the model can predict the predicted similarity score of the two, and adjust the model training parameters according to the difference between the predicted similarity score and the actual similarity score. In this way, the model can accurately predict the similarity between a pair of input vectors contained in the concatenated vector.

[0184] For example, in application, the query vector and the retrieved POI vector may be concatenated to obtain a concatenated vector and input it into a similarity score evaluation model, so that the similarity score evaluation model outputs the similarity score between the query vector and the retrieved POI vector.

[0185] In other examples, the geographic language model can also be used to implement the named entity recognition function, that is, the geographic language model can be used to identify the type of geographic object to which each word in the input keyword belongs. Based on this, a model capable of named entity recognition can be trained based on the aforementioned pre-trained geographic language model. As an example, after the step of obtaining the pre-trained geographic language model, the method can also include:

[0186] Acquire the data representation of the POI, the data representation of the AOI, and the data representation of the Road to construct a second data set;

[0187] A target data representation of a target geographic entity is obtained from the second data set and input into a pre-trained geographic language model, so that the geographic language model predicts a predicted type to which the target data representation belongs based on the input target data representation, and performs training based on the difference between the predicted type and the actual type to which the target geographic entity belongs.

[0188] As an example, the data representation of AOI may include information such as the name, address description text, type, ID, or absolute position in a geographic grid of AOI. The data representation of Road may include information such as the name, address description text, type, ID, or absolute position in a geographic grid of Road.

[0189] In the constructed data set, each piece of data may include the data representation of each geographic entity, and the type of the geographic entity is the label of the data. The data set can be used to train a geographic language model, where the input of the model is the data representation of the target geographic entity, and the output is the type of the geographic entity. Therefore, the model can predict the type of the geographic entity based on the input data representation. Based on the difference between the predicted type and the actual type to which the target geographic entity belongs, the training parameters of the model are adjusted. After the training is completed, a model capable of identifying the type to which the geographic entity belongs can be obtained. Based on this, this embodiment further trains a model capable of named entity recognition based on the pre-trained geographic language model.

[0190] In some examples, the method may further include:

[0191] Get the query text of the geographic entity type to be queried;

[0192] The query information is input into a trained geographic language model to obtain the geographic entity type to which the geographic entity text contained in the query text belongs according to the geographic language model.

[0193] This embodiment can implement a named entity recognition service based on a model capable of performing named entity recognition. There are many sources of query text, such as a client or other services, which are not limited in this embodiment.

[0194] For example, you can input keywords into the model, and the model can segment the input text to obtain one or more words, and identify the geographic entity type to which each word belongs. For example, if you input "Hongcheng Garden A1 Building, Yanyu Road, Ersha Island, Yuexiu District, Guangzhou" into the model, the model can output "Ersha Island Yanyu Road" as the street Road, "Hongcheng Garden" as the AOI, and "Hongcheng Garden A1 Building" as the POI.

[0195] like Figure 2DAs shown, it is a schematic diagram of another user interface shown in this specification according to an exemplary embodiment. In the field of instant delivery, the service provider can be configured with a delivery time estimation model. The model needs to estimate the delivery time for the delivery capacity to deliver the items to the user's delivery address after the user places an order. The input of the delivery time estimation model includes the POI of the current location of the delivery capacity, the POI of the merchant address where the order is placed, and the POI of the delivery address. The vectors of these three POIs can be obtained by the geographic language model of this embodiment. For example, the POI vectors of these three POIs can be queried from the vector database, so that the accurate estimated delivery time can be obtained based on the delivery time estimation model, such as the estimated delivery time shown in the user interface in the figure.

[0196] Corresponding to the above-mentioned embodiment of the method for processing a geographic language model, this specification also provides an embodiment of a device for processing a geographic language model and a computer device used therein.

[0197] The embodiments of the processing device of the geographic language model in this specification can be applied to computer devices, such as servers or terminal devices, etc. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 3 The figure is a hardware structure diagram of the computer device where the processing device of the geographic language model of this specification is located, except Figure 3 In addition to the processor 310, memory 330, network interface 320, and non-volatile memory 340 shown, the computer device where the processing device of the geographic language model is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0198] like Figure 4 As shown, Figure 4 is a block diagram of a processing device for a geographic language model according to an exemplary embodiment of the present specification, the device comprising:

[0199] The acquisition module 41 is used to: acquire geographic information of the following three types of geographic entities within a preset geographic range: point of interest POI, area AOI and road Road;

[0200] The construction module 42 is used to construct a data representation of each geographic entity according to the geographic information of each geographic entity; wherein the data representation of the POI includes: address description text, longitude and latitude, and association information describing the AOI and / or Road associated with the POI; the association relationship includes: an inclusion relationship representing that the AOI includes the POI, and an adjacency relationship representing that the AOI is adjacent to the POI or the Road is adjacent to the POI;

[0201] The training module 43 is used to train a preset geographic language model based on the data representation of the geographic entity.

[0202] In some examples, the association information includes one or more of the following:

[0203] The ID of the associated geographic entity, the type of the associated geographic entity, the type of the association relationship between the POI and the associated geographic entity, the absolute position of the associated geographic entity in the geographic grid, and the distance information between the POI and the associated geographic entity;

[0204] The associated geographic entity includes the AOI and / or Road having the associated relationship with the POI.

[0205] In some examples, the association information includes an ID of an associated geographic entity; the ID of each AOI and / or Road within the preset geographic range is determined in the following manner:

[0206] Dividing the preset geographical scope into multiple administrative levels, and configuring corresponding administrative unit IDs for each administrative unit in each administrative level;

[0207] Numbering each associated geographic entity contained in the lowest administrative level in the multiple administrative levels;

[0208] The administrative unit ID of each administrative level to which the associated geographic entity belongs and the number are combined to obtain the ID of the associated geographic entity.

[0209] In some examples, the absolute position of the associated geographic entity in the geographic grid is determined as follows:

[0210] Divide the preset geographical range into a plurality of rectangular geographical grids according to rows and columns, and configure a row number and a column number for each geographical grid; wherein the row direction is the latitude direction, and the column direction is the longitude direction;

[0211] Obtain the row number and column number of the geographic network where the lower left corner of the associated geographic entity is located, as well as the row number and column number of the geographic network where the lower right corner of the associated geographic entity is located, and use the obtained numbers as the absolute position of the associated geographic entity in the geographic grid; wherein the lower left corner refers to the southwesternmost point of the associated geographic entity on the map, and the upper right corner refers to the northeasternmost point of the associated geographic entity on the map.

[0212] In some examples, the distance information between the POI and the associated geographic entity is determined by:

[0213] A first distance between the POI and the lower left corner of the associated geographic entity and a second distance between the POI and the upper right corner of the associated geographic entity are calculated, and distance information between the POI and the associated geographic entity is determined based on the first distance and the second distance.

[0214] In some examples, the training module 43 is further used to:

[0215] Constructing an initial geographic language model with initial parameters; the initial geographic language model includes a pre-trained text encoder and a geo encoder; the text encoder is used to encode the address description text in the data representation, and the geo encoder is used to encode the geographic representation in the data representation, and the geographic representation is information in the data representation other than the address description text;

[0216] Performing a training task related to the geographic representation on the initial geographic language model to obtain a pre-trained geographic language model;

[0217] After fixing the parameters of the geocoder in the pre-trained geo-language model, the pre-trained geo-language model is subjected to a training task related to the address description text to obtain a pre-trained geo-language model.

[0218] In some examples, the training task related to the geographic representation includes the following geographic mask modeling task:

[0219] A portion of the geographic representation in the data representation of the POI is masked, and the masked data representation of the POI is input into the geographic language model so that the geographic language model predicts prediction information of the masked position, and is trained based on an error between the prediction information and the masked real information in the data representation of the POI.

[0220] In some examples, the training tasks related to the geographic representation include the following geographic contrast learning tasks:

[0221] The data representations of any two POIs are obtained and input into the geographic language model, so that the geographic language model calculates the vector similarity of the two POIs based on the geographic representations in the data representations of the two POIs, and then trains based on the geographic distance between the longitudes and latitudes in the data representations of the two POIs to minimize the error between the vector similarity and the geographic distance.

[0222] In some examples, the training tasks associated with the geographic representation include a joint task of geographic mask modeling and geographic contrastive learning:

[0223] Get the real data representation of any two POIs;

[0224] Masking parts of the geographic representations in the data representations of the two POIs respectively, and inputting the masked data representations of the two POIs into the geographic language model, so that the geographic language model:

[0225] Predicting the prediction information of the mask position in the mask data representation of the two POIs, after obtaining the predicted data representation of the two POIs, calculating the vector similarity of the two POIs based on the geographic representation in the predicted data representation of the two POIs, and then training based on the geographic distance between the longitude and latitude in the real data representation of the two POIs to minimize the error between the vector similarity and the geographic distance; wherein the geographic representation is other information in the data representation except the address description text.

[0226] In some examples, the training task related to the address description text includes the following language mask modeling task:

[0227] Part of the characters of the geographic description text in the data representation of the POI is masked, and the masked data representation of the POI is input into the geographic language model so that the geographic language model predicts the predicted characters at the masked positions, and is trained based on the errors between the predicted characters and the real characters masked in the data representation of the POI.

[0228] In some examples, the geographic description text includes: a POI name and a POI address;

[0229] The masked characters in the address description text include the POI name or POI address.

[0230] In some examples, the training task related to the address description text includes the following joint task of geographic mask modeling and language mask modeling:

[0231] Masking part of the address description text in the real data representation of the POI, and masking part of the geographical representation in the data representation of the POI, and inputting the masked data representation of the POI into a preset geographic language model, so that the geographic language model:

[0232] The information of the masked address description text in the masked data representation is predicted, and the information of the masked geographic representation in the masked data representation is predicted. After obtaining the predicted data representation of the POI, training is performed to minimize the error between the predicted data representation of the POI and the true data representation.

[0233] In some examples, there are multiple training tasks; in the process of performing the multiple training tasks on the geographic language model, target training tasks are first uniformly sampled from the multiple training tasks to train the geographic language model; after the geographic language model is trained on all training tasks, target training tasks are sampled from the multiple training tasks based on the loss value of the loss function of the training task to train the geographic language model; wherein the loss value is positively correlated with the probability of being sampled.

[0234] In some examples, the text encoder is connected to a first adversarial attention module, which is used to generate a first interference coefficient so that the text encoder encodes the address description text in the data representation based on the first interference system; and / or, the geo encoder is connected to a second adversarial attention module, which is used to generate a second interference coefficient so that the geo encoder encodes the geographic representation based on the second interference system.

[0235] In some examples, the apparatus, including a storage module, is configured to:

[0236] The POI vectors generated by the geographic language model for the data representation of each POI are obtained and stored in a vector database.

[0237] In some examples, the device also includes a retrieval module, which is used to: receive query information related to the POI; input the query information into the geographic language model to obtain a query vector of the query information output by the geographic language model; and retrieve a POI vector similar to the query vector from a preset vector database.

[0238] In some examples, the query information includes: the user's current geographic location information; or, the current search term provided by the user.

[0239] In some examples, the query vector is concatenated with the retrieved POI vector to obtain a concatenated vector and input into a similarity score evaluation model so that the similarity score evaluation model outputs a similarity score between the query vector and the retrieved POI vector;

[0240] The similarity score evaluation model is obtained by training the geographic language model using a spliced ​​vector sample, wherein the spliced ​​vector sample includes: a vector spliced ​​from a historical query vector and a historical clicked POI vector, and a vector spliced ​​from a historical query vector and a historical unclicked POI vector; the historical query vector refers to a vector corresponding to a user's historical search terms, the historical clicked POI vector refers to a vector corresponding to a historical clicked POI clicked by a user after searching for the historical search terms, and the historical unclicked POI vector refers to a vector corresponding to a clicked POI not clicked by a user after searching for the historical search terms

[0241] In some examples, the training module 43 is further used to: obtain the data representation of the POI, the data representation of the AOI, and the data representation of the Road to construct a second data set; obtain the target data representation of the target geographic entity from the second data set and input it into the pre-trained geographic language model, so that the geographic language model predicts the predicted type of the target data representation based on the input target data representation, and performs training based on the difference between the predicted type and the actual type of the target geographic entity.

[0242] In some examples, the apparatus further includes a query module for: obtaining a query text of a geographic entity type to be queried;

[0243] The query information is input into a trained geographic language model to obtain the geographic entity type to which the geographic entity text contained in the query text belongs according to the geographic language model.

[0244] The implementation process of the functions and effects of each module in the above-mentioned processing device of the geographic language model is specifically described in the implementation process of the corresponding steps in the above-mentioned processing method of the geographic language model, and will not be repeated here.

[0245] Accordingly, an embodiment of the present specification also provides a computer program product, including a computer program, which implements the steps of the aforementioned geographic language model processing method embodiment when executed by a processor.

[0246] Correspondingly, an embodiment of the present specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of an embodiment of a method for processing a geographic language model when executing the program.

[0247] Accordingly, an embodiment of the present specification also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the embodiment of the method for processing a geographic language model are implemented.

[0248] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0249] The above embodiments can be applied to one or more computer devices, where the computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0250] The computer device may be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0251] The computer device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0252] The network where the computer device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0253] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0254] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the protection scope of this application.

[0255] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0256] The description of "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0257] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0258] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0259] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for processing a geographic language model, the method comprising: Get the geographic information of the following three types of geographic entities within the preset geographic range: point of interest POI, area AOI and road Road; According to the geographic information of each geographic entity, construct a data representation of each POI; wherein the data representation of the POI includes: address description text, longitude and latitude, and association information describing the AOI and / or Road associated with the POI; the association relationship includes: an inclusion relationship representing that the AOI includes the POI, and an adjacency relationship representing that the AOI is adjacent to the POI or the Road is adjacent to the POI; Based on the data representation of each of the POIs, a preset geographic language model is trained.

2. According to the method of claim 1, the associated information includes one or more of the following: The ID of the associated geographic entity, the type of the associated geographic entity, the type of the association relationship between the POI and the associated geographic entity, the absolute position of the associated geographic entity in the geographic grid, and the distance information between the POI and the associated geographic entity; in, The associated geographic entity includes the AOI and / or Road having the associated relationship with the POI.

3. According to the method of claim 2, the association information includes an ID of an associated geographic entity; the ID of each AOI and / or Road within the preset geographic range is determined by: Dividing the preset geographical scope into multiple administrative levels, and configuring corresponding administrative unit IDs for each administrative unit in each administrative level; Numbering each associated geographic entity contained in the lowest administrative level in the multiple administrative levels; The administrative unit ID of each administrative level to which the associated geographic entity belongs and the number are combined to obtain the ID of the associated geographic entity.

4. The method according to claim 2, wherein the absolute position of the associated geographic entity in the geographic grid is determined by: The preset geographical range is divided into a plurality of rectangular geographical grids according to rows and columns, and a row number and a column number are configured for each geographical grid; wherein, The row direction is the latitude direction, and the column direction is the longitude direction; Obtain the row number and column number of the geographic network where the lower left corner of the associated geographic entity is located, as well as the row number and column number of the geographic network where the upper right corner of the associated geographic entity is located, and use the obtained numbers as the absolute position of the associated geographic entity in the geographic grid; wherein the lower left corner refers to the southwesternmost point of the associated geographic entity on the map, and the upper right corner refers to the northeasternmost point of the associated geographic entity on the map.

5. According to the method of claim 4, the distance information between the POI and the associated geographic entity is determined by: A first distance between the POI and the lower left corner of the associated geographic entity and a second distance between the POI and the upper right corner of the associated geographic entity are calculated, and distance information between the POI and the associated geographic entity is determined based on the first distance and the second distance.

6. According to the method of claim 1, the training of the preset geographic language model comprises: Constructing an initial geographic language model with initial parameters; the initial geographic language model includes a pre-trained text encoder and a geographic encoder; The text encoder is used to encode the address description text in the data representation, and the geo encoder is used to encode the geographic representation in the data representation, wherein the geographic representation is information in the data representation other than the address description text; Performing a training task related to the geographic representation on the initial geographic language model to obtain a pre-trained geographic language model; After fixing the parameters of the geocoder in the pre-trained geo-language model, the pre-trained geo-language model is subjected to a training task related to the address description text to obtain a pre-trained geo-language model.

7. The method according to claim 6, wherein the training task related to the geographic representation comprises the following geographic mask modeling task: A portion of the geographic representation in the data representation of the POI is masked, and the masked data representation of the POI is input into the geographic language model so that the geographic language model predicts prediction information of the masked position, and is trained based on an error between the prediction information and the masked real information in the data representation of the POI.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images