Automatic association processing method based on open source place name data

Abstract

The invention relates to the technical field of place name data processing, and particularly discloses an automatic association processing method based on open source place name data, which comprises the following steps: firstly, carrying out entity detection on natural language text content containing place names to extract candidate place names, and taking other content in the text as supplement; the semantic coding technology based on deep learning is adopted to carry out semantic coding and compensation type interactive fusion on candidate place names and supplementary content thereof, and the supplementary content is used as a context background to optimize semantic feature expression of the candidate place names. And constructing a place name alternative list by querying associated entity data of the candidate place names in the geographic database, and realizing automatic association of the place name data based on semantic similarity between each alternative place name in the list and the candidate place names. According to the application, the accuracy of place name data association can be effectively improved, meanwhile, the dependence on manual annotation is reduced, and the data processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of place name data processing, and more specifically, to an automatic association processing method based on open source place name data. Background Art

[0002] With the rapid development of informatization and digitalization, the application of geographic information system (GIS) has penetrated into every corner of social life. As an important part of geographic information system, the processing and management of place name data plays an irreplaceable role in many fields such as spatial data analysis, location services, navigation systems and intelligent transportation. With the development of the Internet, the amount of natural language text content containing place name information (such as news reports, social media, travel blogs, etc.) has increased dramatically. By accurately identifying and geographic association processing these open source place name data, the accuracy of positioning can be effectively improved. For example, online map platforms can provide more accurate destination guidance through the association of place name data; emergency rescue services can quickly locate the specific location corresponding to the place name through the association of place name data, thereby improving rescue efficiency.

[0003] However, traditional place name data processing methods mainly rely on manual annotation or rule-based matching algorithms. Among them, although manual annotation has high accuracy, it is extremely inefficient. When faced with massive text data, it is costly and time-consuming. Rule-based matching algorithms are relatively efficient, but they are less flexible and difficult to adapt to complex and changeable place name expressions. Especially when processing natural language texts, since place names may appear in various forms, such as abbreviations, aliases, dialect expressions, etc., rule-based matching algorithms often find it difficult to accurately identify and associate them with the correct geographical location.

[0004] Therefore, an optimized automatic association processing method based on open source place name data is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an automatic association processing method based on open source place name data, which first performs entity detection on the natural language text content containing place names to extract candidate place names, and uses other content in the text as a supplement, and uses deep learning-based semantic coding technology to semantically encode and compensate for interactive fusion of candidate place names and their supplementary content, so as to use the supplementary content as a context background to optimize the semantic feature expression of the candidate place name, and then constructs a place name candidate list by querying the associated entity data of the candidate place name in the geographic database, and realizes automatic association of place name data based on the semantic similarity between each candidate place name in the list and the candidate place name, which can effectively improve the accuracy of place name data association, while reducing dependence on manual annotation and improving data processing efficiency.

[0006] According to one aspect of the present application, there is provided an automatic association processing method based on open source place name data, which includes: Get natural language text content containing place names; Performing entity detection on the natural language text content containing place names to extract candidate place names, and defining the content of the natural language text content containing place names other than the candidate place names as place name supplementary context content; Based on the supplementary context content of the place name, the candidate place name is subjected to semantic compensation optimization based on principal component analysis to obtain an optimized semantic embedding coding vector of the candidate place name; Querying the associated entity data of the candidate place name in the geographic database to obtain a candidate list of place names; Performing semantic embedding coding on each candidate place name in the candidate place name list to obtain a sequence of semantic embedding coding vectors of the candidate place names; Based on the semantic similarity between the optimized candidate place name semantic embedding coding vector and each alternative place name semantic embedding coding vector in the sequence of alternative place name semantic embedding coding vectors, an association between the alternative place name and the candidate place name is established.

[0007] Preferably, based on the supplementary context content of the place name, the semantic compensation optimization based on principal component analysis is performed on the candidate place name to obtain an optimized semantic embedding coding vector of the candidate place name, including: Performing semantic embedding coding on the candidate place name to obtain a semantic embedding coding vector of the candidate place name; Performing context semantic coding on the place name supplementary context content to obtain a place name supplementary content context semantic coding vector; The place name supplementary content context semantic coding vector and the candidate place name semantic embedding coding vector are interactively optimized using principal component compensation to obtain the optimized candidate place name semantic embedding coding vector.

[0008] Preferably, context semantic encoding is performed on the place name supplementary context content to obtain a place name supplementary content context semantic encoding vector, including: The place name supplementary context content is contextually semantically encoded using a contextual semantic encoder based on the BERT model to obtain the place name supplementary content contextual semantic encoding vector.

[0009] Preferably, the feature principal component compensation interactive optimization is performed on the place name supplementary content context semantic coding vector and the candidate place name semantic embedding coding vector to obtain the optimized candidate place name semantic embedding coding vector, including: Performing principal component extraction on the place name supplementary content context semantic coding vector and the candidate place name semantic embedding coding vector to obtain a set of place name supplementary content semantic feature principal component coding vectors and a set of candidate place name semantic feature principal component coding vectors; Performing semantic difference significance measurement on the set of principal component coding vectors of semantic features of supplementary content of place names and the set of principal component coding vectors of semantic features of candidate place names to obtain a candidate place name-supplementary content semantic difference embedding compensation coding weight vector; Based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the set of principal component coding vectors of the place name supplementary content semantic features and the set of principal component coding vectors of the candidate place name semantic features are subjected to compensatory aggregation interactive coding to obtain the optimized candidate place name semantic embedding coding vector.

[0010] Preferably, the set of principal component coding vectors of semantic features of the place name supplementary content and the set of principal component coding vectors of semantic features of the candidate place name are measured for semantic difference significance to obtain a candidate place name-supplementary content semantic difference embedding compensation coding weight vector, including: The set of principal component coding vectors of the semantic features of the supplementary content of place names and the set of principal component coding vectors of the semantic features of the candidate place names are constructed into a principal component aggregation coding feature map of the semantic features of the supplementary content of place names and a principal component aggregation coding feature map of the semantic features of the candidate place names; Inputting the principal component aggregation coding feature map of semantic features of supplementary content of place names and the principal component aggregation coding feature map of semantic features of candidate place names into a feature embedding unit respectively to obtain a semantic feature weight vector of supplementary content of place names and a semantic feature weight vector of candidate place names; Based on the place name supplementary content semantic feature weight vector and the candidate place name semantic feature weight vector, the candidate place name-supplementary content semantic difference embedding compensation coding weight vector is calculated.

[0011] Preferably, based on the semantic feature weight vector of the place name supplementary content and the semantic feature weight vector of the candidate place name, the candidate place name-supplementary content semantic difference embedding compensation coding weight vector is calculated, including: The position difference vector between the place name supplementary content semantic feature weight vector and the candidate place name semantic feature weight vector is calculated and the absolute value of the position difference vector is taken to obtain the candidate place name-supplementary content semantic difference embedding compensation coding weight vector.

[0012] Preferably, based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the set of the place name supplementary content semantic feature principal component coding vectors and the set of the candidate place name semantic feature principal component coding vectors are subjected to compensatory aggregation interactive coding to obtain the optimized candidate place name semantic embedding coding vector, including: Calculating the positional mean vector of the set of principal component coding vectors of the semantic features of the supplementary content of the place name and the set of principal component coding vectors of the semantic features of the candidate place names to obtain principal component representation coding vectors of the semantic features of the supplementary content of the place name and principal component representation coding vectors of the semantic features of the candidate place names; Based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the place name supplementary content semantic feature principal component representation coding vector and the candidate place name semantic feature principal component representation coding vector are aggregated and interactively encoded to obtain the optimized candidate place name semantic embedding coding vector.

[0013] Preferably, establishing an association between the candidate place name and the candidate place name based on the semantic similarity between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of the candidate place name semantic embedding coding vectors includes: Calculating the semantic matching degree between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of candidate place name semantic embedding coding vectors to obtain a sequence of semantic matching degrees; A maximum value in the sequence of semantic matching degrees is extracted and it is determined whether the maximum value exceeds a preset threshold. If so, an association is established between the alternative place name corresponding to the maximum value and the candidate place name.

[0014] Preferably, calculating the semantic matching degree between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of candidate place name semantic embedding coding vectors to obtain a sequence of semantic matching degrees includes: The cosine similarity between the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector is calculated as the semantic matching degree.

[0015] This application has at least the following technical effects: Compared with the prior art, the automatic association processing method based on open source place name data provided by the present application first performs entity detection on the natural language text content containing place names to extract candidate place names, and uses other content in the text as a supplement, and uses semantic coding technology based on deep learning to semantically encode and interactively fuse the candidate place names and their supplementary content, so as to use the supplementary content as the context background to optimize the semantic feature expression of the candidate place names, and then construct a place name candidate list by querying the associated entity data of the candidate place name in the geographic database, and realize automatic association of place name data based on the semantic similarity between each candidate place name in the list and the candidate place name. In this way, the accuracy of the association of place name data can be effectively improved, while reducing the dependence on manual annotation and improving data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 The present invention is a flowchart of an automatic association processing method based on open source place name data according to an embodiment of the present application.

[0018] Figure 2 This is a data flow diagram of an automatic association processing method based on open source place name data according to an embodiment of the present application.

[0019] Figure 3 This is a flowchart of sub-step S3 of the automatic association processing method based on open source place name data according to an embodiment of the present application.

[0020] Figure 4 This is a flowchart of sub-step S33 of the automatic association processing method based on open source place name data according to an embodiment of the present application.

[0021] Figure 5 This is a flowchart of sub-step S332 of the automatic association processing method based on open source place name data according to an embodiment of the present application.

[0022] Figure 6 This is a flowchart of sub-step S6 of the automatic association processing method based on open source place name data according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0024] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0025] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] It should be noted that all data acquisition and processing in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the corresponding device owner.

[0028] Specifically, Figure 1 The present invention is a flowchart of an automatic association processing method based on open source place name data according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the automatic association processing method based on open source place name data according to an embodiment of the present application. Figure 1 and Figure 2As shown, the automatic association processing method based on open source place name data includes the steps of: S1, obtaining natural language text content containing place names; S2, performing entity detection on the natural language text content containing place names to extract candidate place names, and defining the content other than the candidate place names in the natural language text content containing place names as place name supplementary context content; S3, based on the place name supplementary context content, performing semantic compensation optimization based on principal component analysis on the candidate place names to obtain optimized candidate place name semantic embedding coding vectors; S4, querying the associated entity data of the candidate place names in the geographic database to obtain a place name candidate list; S5, performing semantic embedding coding on each candidate place name in the place name candidate list to obtain a sequence of candidate place name semantic embedding coding vectors; S6, establishing an association between the candidate place name and the candidate place name based on the semantic similarity between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of candidate place name semantic embedding coding vectors.

[0029] In the above-mentioned automatic association processing method based on open source place name data, the step S1 obtains natural language text content containing place names. In a specific example of the present application, the natural language text content containing place names can come from various open source channels, such as news reports, social media, user comments, travel blogs, etc. In order to extract text fragments containing place name information from various data sources, it is necessary to consider the data characteristics of different sources, and methods such as web crawling, API interface calls, social media platform cooperation, and user-generated content (UGC) collection can be used.

[0030] Specifically, first of all, in terms of web crawling, it mainly relies on advanced web crawler algorithms, which can automatically access web page resources on the Internet according to pre-set rules, and filter and crawl text content containing place name information. Web crawlers will regularly update crawling actions at a specific frequency to ensure that the collected information is kept up to date. In addition, crawlers are also capable of processing dynamically loaded content, and can also effectively capture place name-related information that is asynchronously loaded through scripting languages ​​such as JavaScript.

[0031] Secondly, using API interfaces is another efficient way to obtain text content related to place names. Many online map service providers, news media websites and social platforms have opened official API interfaces, allowing third-party developers to legally obtain specified types of data. By establishing partnerships with these service providers and following the corresponding authorization and certification processes, a large amount of high-quality original data can be easily obtained. For example, some map APIs provide functions such as searching for locations and route planning, which can directly query text descriptions with geographic location tags; while news or social platform APIs help capture mentions of place names in real-time event reports or user comments.

[0032] Furthermore, cooperation with social media platforms constitutes an indispensable data source channel. With the popularity of social applications such as Weibo and WeChat Moments, more and more people tend to share their personal experiences and insights on the Internet, including content involving specific geographical locations. Therefore, reaching cooperation agreements with major social platforms has become one of the key ways to obtain such valuable information. In this process, special attention should be paid to user privacy protection issues to ensure that all operations comply with laws and regulations and data collection work is carried out under the premise of respecting the wishes of users.

[0033] Finally, user-generated content (UGC), as an important source of information that has emerged in recent years, is also highly valued. UGC covers a wide range, from travel blogs, forum posts to text descriptions under photos and videos, which can all be regarded as forms of UGC. For this part of unstructured data rich in place name elements, more flexible and diverse collection methods are often required. On the one hand, special theme communities can be set up to attract the majority of netizens to actively participate in contributing materials; on the other hand, natural language processing technology can be used to automatically screen and classify massive UGC content. In this way, it can not only ensure that the amount of data is large enough, but also improve the quality of information to a certain extent, laying a solid foundation for subsequent in-depth analysis.

[0034] In the above-mentioned automatic association processing method based on open source place name data, the step S2 performs entity detection on the natural language text content containing place names to extract candidate place names, and defines the content other than the candidate place names in the natural language text content containing place names as place name supplementary context content. Specifically, in order to accurately locate the place name entities existing in the text, the present application further uses entity detection (NER, Named Entity Recognition) technology to process the natural language text content containing place names to clarify the place name part in the text, and uses the surrounding text as place name supplementary information to provide contextual basis for the semantic optimization understanding of place name information. In an embodiment of the present application, a conditional random field (CRF) model is used to perform entity detection on the natural language text content containing place names to identify and extract place name entities in the text as candidate place names. Among them, the conditional random field model is a discriminant model based on statistics, which can accurately identify and extract place name entities by comprehensively considering the context information in the input natural language text content.

[0035] Specifically, we first need to build a training dataset specifically for place name recognition. This dataset should include a large number of natural language text samples with place name entities labeled, covering a wide range of language environments and application scenarios, such as news reports, social media, travel blogs, etc. The place names in each sample must be accurately labeled manually or semi-automatically, marking the starting and ending positions, as well as the corresponding place name types (such as city names, country names, street names, etc.), as the basis for training the conditional random field model, so that the model can learn the expression forms and characteristic patterns of place names in different contexts.

[0036] Next, in the model training phase, the conditional random field model was chosen because of its powerful context-awareness and flexible feature engineering support. The conditional random field allows multiple feature functions to be considered simultaneously, not only limited to word-level attributes, but also advanced features such as part-of-speech tagging, dependency syntactic relations, and named entity boundary markers. In addition, the conditional random field can handle label dependency problems in sequence data, that is, the choice of the previous label will affect the probability distribution of the next label, which is crucial for correctly capturing place name entities in continuous text. The model parameters are adjusted through an iterative optimization algorithm to make the prediction results as close to the actual annotations as possible, and finally a place name recognition model with high accuracy and recall is obtained.

[0037] After completing the model training, when entering the actual application stage, facing new natural language text input, the conditional random field model will scan the entire sentence structure word by word, and calculate the possibility of place names appearing in each location based on the preset feature template. When encountering a suspected place name fragment, the model will make a judgment based on the clues provided by the context to decide whether to confirm it as a real place name entity. For words with multiple meanings or uncertainties, such as some words that are both common nouns and proper nouns, the model can also refer to contextual information in a larger range to assist in decision-making. Once it is determined that a place name entity exists at a certain location, the exact boundaries are recorded, and the rest is regarded as supplementary context content for the place name.

[0038] Specifically, special attention should be paid to some special cases during the processing, such as abbreviations, aliases, dialect expressions and other variants of place names. To this end, in addition to relying on the capabilities of the conditional random field model itself, external resource libraries can also be introduced for auxiliary matching. For example, a dictionary containing common place names and their various variants can be established to expand the model's recognition range; or a rule engine can be used to achieve fast searches based on specific patterns, such as the city name corresponding to the postal code. This not only improves recognition efficiency, but also enhances the system's flexibility in dealing with complex and changeable forms of place name expressions.

[0039] In the above-mentioned automatic association processing method based on open source place name data, the step S3 performs semantic compensation optimization based on principal component analysis on the candidate place name based on the supplementary context content of the place name to obtain an optimized candidate place name semantic embedding coding vector. Figure 3 FIG. 4 is a flowchart of sub-step S3 of the automatic association processing method based on open source place name data according to an embodiment of the present application. Figure 3 As shown, the step S3 includes the steps of: S31, performing semantic embedding coding on the candidate place name to obtain a candidate place name semantic embedding coding vector; S32, performing contextual semantic coding on the place name supplementary context content to obtain a place name supplementary content contextual semantic coding vector; S33, performing feature principal component compensation interactive optimization on the place name supplementary content contextual semantic coding vector and the candidate place name semantic embedding coding vector to obtain the optimized candidate place name semantic embedding coding vector.

[0040] Specifically, the step S31 performs semantic embedding coding on the candidate place names to obtain semantic embedding coding vectors of the candidate place names. Specifically, the present application takes into account that traditional place name association methods are often based on character-level matching operations, which has great limitations when dealing with complex and changeable place name expressions. Therefore, in order to improve the accuracy of place name association, the present application further uses a semantic embedding coding technology based on deep learning to process the candidate place names, so as to map the candidate place names to a high-dimensional semantic space and obtain a semantic embedding coding vector of the candidate place names. In an embodiment of the present application, the Word2Vec model is used as a semantic embedding encoder to semantically encode the candidate place names. The Word2Vec model is a commonly used word embedding technology, which can map the candidate place names to a high-dimensional vector space while retaining their original semantic information, so that semantically similar place name information is closer in the vector space, thereby providing a basis for subsequent place name association processing.

[0041] Specifically, in a specific example of the present application, the step S32 includes: using a context semantic encoder based on the BERT model to perform context semantic encoding on the place name supplementary context content to obtain the place name supplementary content context semantic encoding vector. Specifically, since the place name supplementary context content contains the contextual environment information of the place name entity, it plays an important supplementary role in the semantic understanding of the candidate place name. Therefore, the present application further adopts the BERT model to perform context semantic encoding on the place name supplementary context content. Among them, the BERT (Bidirectional Encoder Representations from Transformers) model is a pre-trained language model based on a bidirectional Transformer architecture. It is based on a self-attention mechanism and can effectively capture the semantic dependencies between words in the text by performing bidirectional context understanding and modeling on the place name supplementary context content, thereby providing a more comprehensive and accurate context background for the semantic understanding of the place name entity, and generating a place name supplementary content context semantic encoding vector.

[0042] Specifically, the step S33 performs feature principal component compensatory interactive optimization on the contextual semantic coding vector of the place name supplementary content and the candidate place name semantic embedding coding vector to obtain the optimized candidate place name semantic embedding coding vector. Specifically, in order to make full use of the contextual semantic background information of the place name supplementary content, the present application adopts a compensatory interactive fusion method based on feature principal components, which performs principal component feature extraction and information complementary analysis on the contextual semantic coding vector of the place name supplementary content and the candidate place name semantic embedding coding vector to achieve deep semantic interaction and fusion between the two, thereby obtaining a more accurate and comprehensive semantic expression of the candidate place name. Figure 4 FIG. 4 is a flowchart of sub-step S33 of the automatic association processing method based on open source place name data according to an embodiment of the present application. Figure 4 As shown, the step S33 includes the steps of: S331, performing principal component extraction on the context semantic coding vector of the place name supplementary content and the semantic embedding coding vector of the candidate place name to obtain a set of principal component coding vectors of place name supplementary content semantic features and a set of principal component coding vectors of candidate place name semantic features; S332, performing semantic difference significance measurement on the set of principal component coding vectors of place name supplementary content semantic features and the set of principal component coding vectors of candidate place name semantic features to obtain a candidate place name-supplementary content semantic difference embedding compensation coding weight vector; S333, based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, performing compensatory aggregation interactive coding on the set of principal component coding vectors of place name supplementary content semantic features and the set of principal component coding vectors of candidate place name semantic features to obtain the optimized candidate place name semantic embedding coding vector.

[0043] More specifically, the step S331 is expressed by the formula: ; ; in, represents the contextual semantic encoding vector of the place name supplementary content, represents the semantic embedding encoding vector of the candidate place name, represents the principal component analysis function, represents the transpose of a vector, For the and stated The characteristic scale value of and Respectively indicate the and stated The covariance matrix of Indicates that by The matrix composed of the set of principal component coding vectors of semantic features of supplementary content of place names obtained by eigenvalue decomposition is Indicates that by The diagonal matrix composed of the set of eigenvalues ​​of the semantic principal component of the place name supplementary content obtained by eigenvalue decomposition is Indicates the semantic principal component eigenvalues ​​of the supplementary content of each place name, is the number of eigenvalues ​​of the semantic principal component of the supplementary content of the place name, represents each principal component coding vector of the semantic features of the supplementary content of the place name in the set of principal component coding vectors of the semantic features of the supplementary content of the place name, Indicates that by The matrix composed of the set of principal component encoding vectors of the semantic features of candidate place names obtained by eigenvalue decomposition, Indicates that by The diagonal matrix composed of the set of semantic principal component eigenvalues ​​of candidate place names obtained by eigenvalue decomposition is Represents the semantic principal component eigenvalues ​​of each candidate place name, Represents each candidate place name semantic feature principal component coding vector in a set of candidate place name semantic feature principal component coding vectors.

[0044] Specifically, the principal component analysis (PCA) method is used to mine the main semantic feature components in place name entities and place name supplementary content, and multiple main feature components are extracted from the context semantic coding vector of the place name supplementary content and the semantic embedding coding vector of the candidate place name to form a set of place name supplementary content semantic feature principal component coding vectors and a set of candidate place name semantic feature principal component coding vectors, so as to retain the core semantic information in place name entities and place name supplementary content, while removing redundancy and noise, making subsequent semantic interaction processing more efficient and accurate.

[0045] Figure 5 FIG. 4 is a flowchart of sub-step S332 of the automatic association processing method based on open source place name data according to an embodiment of the present application. Figure 5 As shown, the step S332 includes the steps of: S3321, constructing the set of principal component coding vectors of the semantic features of the place name supplementary content and the set of principal component coding vectors of the semantic features of the candidate place names into a principal component aggregation coding feature map of the semantic features of the place name supplementary content and a principal component aggregation coding feature map of the semantic features of the candidate place names; S3322, inputting the principal component aggregation coding feature map of the semantic features of the place name supplementary content and the principal component aggregation coding feature map of the semantic features of the candidate place names into a feature embedding unit respectively to obtain a place name supplementary content semantic feature weight vector and a candidate place name semantic feature weight vector; S3323, calculating the candidate place name-supplementary content semantic difference embedding compensation coding weight vector based on the place name supplementary content semantic feature weight vector and the candidate place name semantic feature weight vector.

[0046] In a specific example of the present application, step S3321 is expressed by the formula: ; ; in, and They represent the principal component aggregation coding feature map of the semantic features of the supplementary content of place names and the principal component aggregation coding feature map of the semantic features of candidate place names, respectively. Indicates feature reshape.

[0047] Specifically, the set of principal component coding vectors of the semantic features of the place name supplementary content and the set of principal component coding vectors of the semantic features of the candidate place names are further reconstructed into feature graph forms respectively, so as to maintain the structural relationship between the place name entity and the place name supplementary content in the semantic space, and form a principal component aggregation coding feature graph of the semantic features of the place name supplementary content and a principal component aggregation coding feature graph of the semantic features of the candidate place names.

[0048] In a specific example of the present application, step S3322 is expressed by the formula: ; in, represents the mean pooling function, and Respectively indicate the Two learnable weight parameter matrices, represents the Sigmoid activation function, and Respectively indicate the Two learnable weight parameter matrices, represents the normalization function, represents the semantic feature weight vector of the supplementary content of place names, Represents the semantic feature weight vector of candidate place names.

[0049] Specifically, a feature embedding unit is used to perform low-dimensional embedding encoding on the principal component aggregation coding feature map of the semantic features of the place name supplementary content and the principal component aggregation coding feature map of the semantic features of the candidate place names, so as to quantify the feature importance of the semantic principal component features of the place name supplementary content and the semantic principal component features of the candidate place names through processing of a multi-layer neural network, and generate a place name supplementary content semantic feature weight vector and a candidate place name semantic feature weight vector.

[0050] In a specific example of the present application, the step S3323 includes: calculating the position difference vector between the semantic feature weight vector of the place name supplementary content and the semantic feature weight vector of the candidate place name and taking the absolute value of the position difference vector to obtain the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, which is expressed by the formula: ; in, represents the difference operation, Represents the candidate place name-supplementary content semantic difference embedding compensation encoding weight vector, Indicates taking the absolute value.

[0051] Among them, by performing differential operations on the semantic feature weight vector of the place name supplementary content and the semantic feature weight vector of the candidate place name, the information complementarity between the place name entity and the place name supplementary content can be revealed, and a candidate place name-supplementary content semantic difference embedding compensation coding weight vector can be generated to guide subsequent semantic fusion processing.

[0052] More specifically, in a specific example of the present application, the step S333 includes: first, calculating the positional mean vector of the set of principal component coding vectors of the semantic features of the place name supplementary content and the set of principal component coding vectors of the semantic features of the candidate place name to obtain the principal component representation coding vector of the semantic features of the place name supplementary content and the principal component representation coding vector of the semantic features of the candidate place name, which is expressed by the formula: ; ; in, The first one in the set of principal component encoding vectors of the semantic features of the supplementary content of the place name principal component encoding vector of semantic features of supplementary content of place names, The principal component representation encoding vector representing the semantic features of the supplementary content of the place name, The first one in the set of principal component encoding vectors of the semantic features of the candidate place names is represented The principal component encoding vectors of semantic features of candidate place names, Represents the principal component representation encoding vector of the semantic features of the candidate place names.

[0053] Specifically, in the subsequent semantic information compensation interaction, in order to comprehensively consider the main components of the semantic features of the candidate place names and the main components of the semantic features of the place name supplementary content, the principal component features of the set of principal component encoding vectors of the place name supplementary content and the set of principal component encoding vectors of the semantic features of the candidate place names are first integrated by calculating the positional mean.

[0054] Then, based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the place name supplementary content semantic feature principal component representation coding vector and the candidate place name semantic feature principal component representation coding vector are aggregated and interactively encoded to obtain the optimized candidate place name semantic embedding coding vector, which is expressed as follows: ; in, represents dot product, represents the point convolution operation, Represents the optimized candidate place name semantic embedding encoding vector.

[0055] Among them, the candidate place name-supplementary content semantic difference embedding compensation coding weight vector is used as a compensation guide, and the integrated candidate place name semantic feature main component and place name supplementary content semantic feature main component are weighted fused, and through point convolution coding and activation function processing, the key information in the fused feature is further extracted and emphasized to obtain the optimized candidate place name semantic embedding coding vector. In this process, the semantic difference information between the place name supplementary content and the place name entity is effectively identified and compensated by weighted fusion, thereby effectively enhancing the integrity and accuracy of the semantic expression of the candidate place name, which helps to provide a more reliable and accurate semantic information basis for subsequent place name association processing.

[0056] In the above-mentioned automatic association processing method based on open source place name data, the step S4 queries the associated entity data of the candidate place name in the geographic database to obtain a place name candidate list. Specifically, a large amount of place name entity data and related information are stored in the geographic database. By querying the candidate place name in the geographic database, place name entity data that is literally similar or related to the candidate place name can be quickly obtained to form a place name candidate list, thereby providing rich reference information for subsequent place name association processing.

[0057] Specifically, first of all, at the level of geographic database design, building an efficient data index structure is the basis for ensuring fast query response. Geographic databases usually establish full-text indexes or inverted indexes for place name fields, so that the system can quickly locate entities that are highly relevant to the input place names in massive records. In addition, considering that place names may exist in many variant forms, such as different language versions, historical name changes, etc., additional auxiliary indexes need to be created to support matching operations in a wider range. For example, for some cities with a long history, the database not only includes modern standard names, but also preserves various names recorded in ancient documents, in order to cope with various complex query needs.

[0058] In order to further improve the matching accuracy, fuzzy matching algorithms play an important role. Traditional string comparison-based methods are difficult to cope with the diversity of place name expressions that are prevalent in the real world, so advanced algorithms such as edit distance and phonetic code conversion are introduced to solve such challenges. The edit distance algorithm measures the similarity by calculating the minimum number of changes (insertion, deletion, and replacement of characters) between two strings; while phonetic code conversion converts text into pronunciation or shape feature codes, thereby achieving effective recognition of homophones and similar characters. These algorithms work together to find the corresponding geographical entities even when faced with abbreviations, aliases, and even place names expressed in dialects.

[0059] In the above-mentioned automatic association processing method based on open source place name data, the step S5 performs semantic embedding coding on each alternative place name in the place name alternative list to obtain a sequence of semantic embedding coding vectors of the alternative place names. Specifically, in order to achieve accurate association at the semantic level between each alternative place name in the place name alternative list and the candidate place name, the present application further processes each alternative place name in the place name alternative list by adopting semantic embedding coding technology (such as Word2Vec model), maps each alternative place name to a high-dimensional vector space, and generates a sequence of semantic embedding coding vectors of the alternative place names, so as to calculate and compare semantic similarity based on the distance relationship between each place name entity data in the vector space, thereby establishing accurate semantic association between the candidate place name and the alternative place name.

[0060] In the above-mentioned automatic association processing method based on open source place name data, the step S6 establishes an association between the candidate place name and the candidate place name based on the semantic similarity between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of the candidate place name semantic embedding coding vector. Figure 6 FIG. 6 is a flowchart of sub-step S6 of the automatic association processing method based on open source place name data according to an embodiment of the present application. Figure 6 As shown, the step S6 includes the steps of: S61, calculating the semantic matching degree between the optimized candidate place name semantic embedding coding vector and each alternative place name semantic embedding coding vector in the sequence of alternative place name semantic embedding coding vectors to obtain a sequence of semantic matching degrees; S62, extracting the maximum value in the sequence of semantic matching degrees and determining whether the maximum value exceeds a preset threshold, and if so, establishing an association between the alternative place name corresponding to the maximum value and the candidate place name.

[0061] Specifically, in a specific example of the present application, the step S61 includes: calculating the cosine similarity between the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector as the semantic matching degree. Specifically, cosine similarity is an indicator that measures the directional similarity between two vectors, and its value range is [-1, 1]. The closer the value is to 1, the closer the directions of the two vectors are, that is, the higher the semantic similarity. By using cosine similarity as a measure of the semantic matching degree between the candidate place name and each alternative place name, the degree of semantic association between the candidate place name and each alternative place name can be intuitively reflected.

[0062] In a preferred example of the present application, since the optimized candidate place name semantic embedding coding vector represents the semantic coding features of the candidate place name that is semantically context optimized based on the supplementary context content of the place name, and the alternative place name semantic embedding coding vector represents the semantic coding features of the alternative place name, when calculating the semantic matching degree between them to map them toward a common semantic matching degree space, there will be insufficient correlation correspondence between the semantic coding features of different orders under the different source semantic representations, which will cause a sparse correlation mapping of the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector toward the common semantic matching degree space, thereby reducing the accuracy of the calculated semantic matching degree due to the lack of common mapping inference degree.

[0063] Based on this, before calculating the semantic matching degree between the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector, the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector are firstly jointly associated and optimized, specifically including the steps of: Firstly, the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector are cascaded into a candidate place name-alternative place name semantic information common coding vector; Secondly, based on the feature value of the candidate place name-alternative place name semantic information common coding vector Distance and distance matrix of candidate place names and alternative place names semantic information and the common distance matrix of candidate place name-alternative place name semantic information ; Then, the weighted sum of the candidate place name-alternative place name semantic information common distance matrix and the candidate place name-alternative place name semantic information common distance matrix is ​​calculated to obtain the candidate place name-alternative place name semantic information common joint distance matrix, which is expressed as follows: ; in, Represents the common distance matrix of candidate place name-alternative place name semantic information, Represents the common distance matrix of candidate place name-alternative place name semantic information, It means point multiplication by position. represents matrix addition, and Represent different weight parameters respectively. Represents the common joint distance matrix of semantic information of candidate place names and alternative place names; Next, the eigenvalues ​​of the candidate place name-alternative place name semantic information joint distance matrix are determined. , and form the candidate place name-alternative place name semantic information joint distance eigenvector ; Then, the candidate place name-alternative place name semantic information common encoding vector as a row vector is matrix multiplied with the candidate place name-alternative place name semantic information common-distance matrix to obtain the candidate place name-alternative place name semantic information common-distance query vector, which is expressed as follows: ; in, Represents the joint encoding vector of the semantic information of candidate place names and alternative place names, represents matrix multiplication, Represents the common distance query vector of candidate place name-alternative place name semantic information; Next, the candidate place name-alternative place name semantic information common two-distance matrix and the candidate place name-alternative place name semantic information common encoding vector autocorrelation matrix are matrix-multiplied to obtain the candidate place name-alternative place name semantic information common two-distance correlation matrix, which is expressed as follows: ; in, represents the transpose of a vector, Represents the common two-distance association matrix of semantic information of candidate place names and alternative place names; Then, after matrix multiplication of the candidate place name-alternative place name semantic information common distance query vector and the candidate place name-alternative place name semantic information common distance association matrix, it is further dot-multiplied with the candidate place name-alternative place name semantic information common joint distance eigenvector to obtain the optimized candidate place name-alternative place name semantic information common encoding vector, which is expressed as follows: ; in, Represents the common joint distance eigenvector of the semantic information of candidate place names and alternative place names, Represents the optimized candidate place name-alternative place name semantic information joint encoding vector; Finally, the optimized candidate place name-alternative place name semantic information joint coding vector is split into an optimized optimized candidate place name semantic embedding coding vector and an optimized alternative place name semantic embedding coding vector.

[0064] Specifically, the first distance matrix and the second distance matrix of the candidate place name-alternative place name semantic information common coding vector obtained by concatenating the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector are used as the fine-grained metric association cluster representation of the candidate place name-alternative place name semantic information common coding vector, and the dynamic programming of the relationship between association clusters of different association clusters is performed on the self-association representation of the candidate place name-alternative place name semantic information common coding vector and the candidate place name-alternative place name semantic information common coding vector respectively to simulate the sparse activation based on neuron clusters of the association system, and the fine-grained predictable sparsity of the candidate place name-alternative place name semantic information common coding vector is coordinated with the intrinsic representation of the metric association cluster of the first distance matrix and the second distance matrix, so as to avoid the lack of common mapping inference degree affected by insufficient association caused by sparsity, and improve the calculation accuracy of the semantic matching degree between the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector.

[0065] Specifically, the step S62 extracts the maximum value in the sequence of the semantic matching degree and determines whether the maximum value exceeds a preset threshold value. If so, an association is established between the alternative place name corresponding to the maximum value and the candidate place name. Specifically, the alternative place name corresponding to the maximum value in the sequence of the semantic matching degree represents the place name entity that is semantically closest or similar to the candidate place name. When its semantic matching degree exceeds the preset threshold value, it can be considered that there is a significant semantic association between the alternative place name and the candidate place name, so that it is determined as an associated place name of the candidate place name, and an association between the two is established, thereby providing an important reference basis for subsequent tasks such as place name resolution, place name standardization or place name association analysis. If the maximum value does not exceed the preset threshold value, it indicates that the semantic association between the candidate place name and all the alternative place names is not significant enough. At this time, it can be considered that the place name resolution or association task has failed, or further processing measures need to be taken, such as expanding the scope of the place name alternative list, further revising the candidate place name or manually reviewing it, so as to improve the accuracy and reliability of the place name resolution and association tasks.

[0066] In summary, the automatic association processing method based on open source place name data based on the embodiment of the present application is explained, which first performs entity detection on the natural language text content containing place names to extract candidate place names, and uses other content in the text as a supplement, and uses deep learning-based semantic coding technology to semantically encode and compensate for interactive fusion of candidate place names and their supplementary content, so as to use the supplementary content as a contextual background to optimize the semantic feature expression of the candidate place name, and then constructs a place name candidate list by querying the associated entity data of the candidate place name in the geographic database, and realizes automatic association of place name data based on the semantic similarity between each alternative place name in the list and the candidate place name. In this way, the accuracy of the association of place name data can be effectively improved, while reducing the dependence on manual annotation and improving data processing efficiency.

[0067] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0068] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0069] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0070] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0071] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An automatic association processing method based on open source place name data, characterized in that: include: Get natural language text content containing place names; Performing entity detection on the natural language text content containing place names to extract candidate place names, and defining the content of the natural language text content containing place names other than the candidate place names as place name supplementary context content; Based on the supplementary context content of the place name, the candidate place name is subjected to semantic compensation optimization based on principal component analysis to obtain an optimized semantic embedding coding vector of the candidate place name; Querying the associated entity data of the candidate place name in the geographic database to obtain a candidate list of place names; Performing semantic embedding coding on each candidate place name in the candidate place name list to obtain a sequence of semantic embedding coding vectors of the candidate place names; Based on the semantic similarity between the optimized candidate place name semantic embedding coding vector and each alternative place name semantic embedding coding vector in the sequence of alternative place name semantic embedding coding vectors, an association between the alternative place name and the candidate place name is established.

2. The automatic association processing method based on open source place name data according to claim 1 is characterized in that: Based on the supplementary context content of the place name, the candidate place name is subjected to semantic compensation optimization based on principal component analysis to obtain an optimized semantic embedding coding vector of the candidate place name, including: Performing semantic embedding coding on the candidate place name to obtain a semantic embedding coding vector of the candidate place name; Performing context semantic coding on the place name supplementary context content to obtain a place name supplementary content context semantic coding vector; The place name supplementary content context semantic coding vector and the candidate place name semantic embedding coding vector are interactively optimized using principal component compensation to obtain the optimized candidate place name semantic embedding coding vector.

3. The automatic association processing method based on open source place name data according to claim 2 is characterized in that: Performing context semantic coding on the place name supplementary context content to obtain a place name supplementary content context semantic coding vector includes: The place name supplementary context content is contextually semantically encoded using a contextual semantic encoder based on the BERT model to obtain the place name supplementary content contextual semantic encoding vector.

4. The automatic association processing method based on open source place name data according to claim 3 is characterized in that: Performing feature principal component compensation interactive optimization on the place name supplementary content context semantic coding vector and the candidate place name semantic embedding coding vector to obtain the optimized candidate place name semantic embedding coding vector, including: Performing principal component extraction on the place name supplementary content context semantic coding vector and the candidate place name semantic embedding coding vector to obtain a set of place name supplementary content semantic feature principal component coding vectors and a set of candidate place name semantic feature principal component coding vectors; Performing semantic difference significance measurement on the set of principal component coding vectors of semantic features of supplementary content of place names and the set of principal component coding vectors of semantic features of candidate place names to obtain a candidate place name-supplementary content semantic difference embedding compensation coding weight vector; Based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the set of principal component coding vectors of the place name supplementary content semantic features and the set of principal component coding vectors of the candidate place name semantic features are subjected to compensatory aggregation interactive coding to obtain the optimized candidate place name semantic embedding coding vector.

5. The automatic association processing method based on open source place name data according to claim 4 is characterized in that: The semantic difference significance measurement is performed on the set of principal component coding vectors of the semantic features of the supplementary content of the place name and the set of principal component coding vectors of the semantic features of the candidate place name to obtain the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, including: The set of principal component coding vectors of the semantic features of the supplementary content of place names and the set of principal component coding vectors of the semantic features of the candidate place names are constructed into a principal component aggregation coding feature map of the semantic features of the supplementary content of place names and a principal component aggregation coding feature map of the semantic features of the candidate place names; Inputting the principal component aggregation coding feature map of semantic features of supplementary content of place names and the principal component aggregation coding feature map of semantic features of candidate place names into a feature embedding unit respectively to obtain a semantic feature weight vector of supplementary content of place names and a semantic feature weight vector of candidate place names; Based on the place name supplementary content semantic feature weight vector and the candidate place name semantic feature weight vector, the candidate place name-supplementary content semantic difference embedding compensation coding weight vector is calculated.

6. The automatic association processing method based on open source place name data according to claim 5 is characterized in that: Calculating the candidate place name-supplementary content semantic difference embedding compensation coding weight vector based on the place name supplementary content semantic feature weight vector and the candidate place name semantic feature weight vector, including: The position difference vector between the place name supplementary content semantic feature weight vector and the candidate place name semantic feature weight vector is calculated and the absolute value of the position difference vector is taken to obtain the candidate place name-supplementary content semantic difference embedding compensation coding weight vector.

7. The automatic association processing method based on open source place name data according to claim 6 is characterized in that: Based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the set of the place name supplementary content semantic feature principal component coding vectors and the set of the candidate place name semantic feature principal component coding vectors are subjected to compensatory aggregation interactive coding to obtain the optimized candidate place name semantic embedding coding vector, including: Calculating the positional mean vector of the set of principal component coding vectors of the semantic features of the supplementary content of the place name and the set of principal component coding vectors of the semantic features of the candidate place names to obtain principal component representation coding vectors of the semantic features of the supplementary content of the place name and principal component representation coding vectors of the semantic features of the candidate place names; Based on the candidate place name-supplementary content semantic difference embedding compensation coding weight vector, the place name supplementary content semantic feature principal component representation coding vector and the candidate place name semantic feature principal component representation coding vector are aggregated and interactively encoded to obtain the optimized candidate place name semantic embedding coding vector.

8. The automatic association processing method based on open source place name data according to claim 7 is characterized in that: Based on the semantic similarity between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of the candidate place name semantic embedding coding vectors, establishing an association between the candidate place name and the candidate place name, including: Calculating the semantic matching degree between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of candidate place name semantic embedding coding vectors to obtain a sequence of semantic matching degrees; A maximum value in the sequence of semantic matching degrees is extracted and it is determined whether the maximum value exceeds a preset threshold. If so, an association is established between the alternative place name corresponding to the maximum value and the candidate place name.

9. The automatic association processing method based on open source place name data according to claim 8 is characterized in that: Calculating the semantic matching degree between the optimized candidate place name semantic embedding coding vector and each candidate place name semantic embedding coding vector in the sequence of candidate place name semantic embedding coding vectors to obtain a sequence of semantic matching degrees, including: The cosine similarity between the optimized candidate place name semantic embedding coding vector and the alternative place name semantic embedding coding vector is calculated as the semantic matching degree.

Images