An error-adaptive deep learning address semantic matching method and system
By combining BERT, Bi-LSTM and Transformer models, address storage errors are adaptively corrected, and the problem of inefficient address matching in the prior art is solved, and efficient and accurate address query and database optimization are achieved.
Patent Information
- Application Number
- CN202211474657.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-23
- Publication Date
- 2025-08-29
- Estimated Expiration
- 2042-11-23
AI Technical Summary
Existing address matching techniques are difficult to match accurately when facing data storage errors, and existing text matching methods cannot effectively identify semantic slight differences, resulting in inefficient query and stress on the database.
Deep learning methods combined with BERT, Bi-LSTM and Transformer models are adopted to obtain the address key element information through word segmentation and word embedding, query statements are constructed and database queries are optimized using memory pools to adaptively correct storage errors.
It improves the accuracy and efficiency of address matching, reduces the number of database queries, reduces the query time and pressure, and enhances the model's migration and interpretability.
Smart Images
Figure CN115718849B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of natural language processing, and specifically relates to an error-adaptive deep learning address semantic matching method and system. Background Art
[0002] In real-world applications, a large amount of address data is collected from the internet and stored in databases. Although data is cleansed before being written to the database, some address data inevitably escapes cleansing rules, leading to data field misalignment. This issue can lead to unforeseen data storage errors in the address data in the database.
[0003] Standard address data is generally stored in a semi-structured form. For example, in an address, there is structured data with each column storing separately such as province, city, and district; there is also unstructured data stored by combining township, village, town, street, and house number.
[0004] Storage errors in address data primarily occur in the structured portion of the data (for example, a "city" column may contain information at the "village" level). When querying addresses in a database, the database query engine primarily relies on the structured portion of the address data (which has the fastest query speed) for data filtering and querying. The unstructured portion of the address data only provides auxiliary query functionality (the more the unstructured portion is used, the slower the query speed). If a storage error occurs in the structured portion of the address, models that implement address matching based on database queries will have difficulty correctly retrieving the data in the database.
[0005] Models that implement matching for database queries can use recursive algorithms to exhaustively search for all address queries to maximize the probability of retrieving the correct data. However, this approach is prohibitively time-consuming in real-world applications, and multiple I / O operations can create read and write pressure on the database. If the address database adopts a distributed architecture, this can also lead to significant peak pressure on network bandwidth.
[0006] Due to the aforementioned difficulties, existing address matching methods tend to use text pairs to complete address matching. However, this approach has the following objective drawbacks: 1. Text pair matching methods determine whether address texts match by comparing global address semantics. This makes it difficult for the model to understand subtle differences in semantics, resulting in a large number of matching errors in difficult negative examples. 2. Text pair matching methods are sensitive to text length and have difficulty correctly determining whether addresses with significantly different lengths match. 3. Related address matching research focuses more on improving text pair matching accuracy and ignores text pair construction methods, which reduces the feasibility of practical model deployment and application. 4. Text pair matching methods strongly rely on front-end word embedding models, which limits the transferability of related models and increases the difficulty of updating them. 5. The overall matching process of text pair matching methods lacks interpretability. These shortcomings make text pair matching methods difficult to implement in practical model deployment and real-world applications. Summary of the Invention
[0007] In response to the shortcomings of the existing technology, the present invention proposes an error-adaptive deep learning address semantic matching method and system. The method includes: a user inputs a non-standard address text into a trained deep learning-based address semantic matching model to obtain a standard address matching result corresponding to the non-standard address;
[0008] The process of processing non-standard address text using the trained deep learning-based address semantic matching model includes:
[0009] S1: Use the BERT model to encode the non-standard address text to obtain the word segmentation result and word embedding vector of the non-standard address;
[0010] S2: The word segmentation results and word embedding vectors are simultaneously input into the Bi-LSTM model and the Transformer model to obtain the address key element information and the judgment result of whether the address key element information is available;
[0011] S3: Use the predict algorithm to construct a query statement based on the available address key element information to perform address query and obtain the standard address matching result corresponding to the non-standard address text.
[0012] Preferably, the process of obtaining address key element information includes: the Bi-LSTM model determines whether each word in the non-standard address text is an address key element based on the word segmentation result and the word embedding vector. If so, the parent category of the address key element is determined to obtain the address key element information, namely the address key element and its corresponding parent category.
[0013] Preferably, the process of determining whether the address key elements are available includes: the Transformer model determines whether the address key element information can be used to construct an address query statement based on the current data storage status of the database, the word segmentation results and word embedding vectors output by the BERT model.
[0014] Preferably, the process of performing address query using the predict algorithm includes:
[0015] The predict algorithm constructs an address query statement based on the available address key element information. The position of the address key element in the address query statement is determined by the parent category of the address key element output by the Bi-LSTM model.
[0016] Check whether there is a standard address matching result corresponding to the address query statement in the memory pool. If so, return the standard address matching result directly. Otherwise, use the address query statement to query the database. If the standard address is found, return the result and write it to the memory pool for subsequent queries.
[0017] Preferably, the process of training the deep learning-based address semantic matching model includes:
[0018] Obtain labeled address text training data and input the address text training data into the BERT model to obtain the word segmentation results and word embedding vectors of the address text;
[0019] The word segmentation results and word embedding vectors are simultaneously input into the Bi-LSTM model and the Transformer model to obtain the address key element information and the discrimination results of whether the address key element information is available;
[0020] Adopting the train algorithm, the address query statement is recursively constructed based on the available address key element information to perform address query and obtain the standard address matching result corresponding to the address text;
[0021] The model parameters are adjusted according to the standard address matching results and labels to obtain a trained address semantic matching model based on deep learning.
[0022] Furthermore, the process of using the train algorithm to construct an address query statement for address query includes: the train algorithm gradually appends address key elements to the address query statement through a recursive method, thereby continuously narrowing the scope of the address query; when there is only one standard address data in the query result, the standard address is the standard address matching result corresponding to the address text.
[0023] Furthermore, the process of training the address semantic matching model based on deep learning also includes: after the train algorithm obtains the standard address matching result corresponding to the address text, it returns the address key elements used to the Transformer model, and at the same time stores the query result, that is, the standard address matching result, into the memory pool. The Transformer model corrects its own judgment result on whether the address key element information is available based on the returned address key elements, and adjusts its own parameters based on the corrected judgment result.
[0024] An error-adaptive deep learning address semantic matching system, comprising: a BERT-based address text input module, a Bi-LSTM module, a Transformer module, and an address matching module;
[0025] The BERT-based address text input module is used to perform text encoding on the address text to obtain the word segmentation result and word embedding vector of the address text;
[0026] The Bi-LSTM module is used to obtain address key element information and determine the position of address key elements in the query statement based on the word segmentation results and word embedding vectors of the address text;
[0027] The Transformer module is used to determine whether the address key element information can be used to construct an address query statement based on the word segmentation results and word embedding vectors of the address text;
[0028] The address matching module is used to construct an address query statement and perform address query based on the output information of the Bi-LSTM module and the Transformer module, and output the standard address matching result of the address text.
[0029] The beneficial effects of the present invention are: 1. By using address elements for step-by-step query, subtle differences in address semantics can be specifically identified; 2. By extracting key elements of the address from the text, the model is desensitized to the data length; 3. When querying the database, only non-standard addresses need to be used, and there is no need to spend a lot of energy on text pair construction; 4. The word embedding model is integrated to further improve the cohesion of the model; 5. The model has learned about the internal errors of the database through training, which can avoid address matching failures caused by database errors; 6. With the help of database caching technology, the model only performs one database query operation for each identical address query statement, thereby greatly reducing the prediction time of the address semantic matching model and improving the model deployment performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 A schematic diagram of the structure of the address semantic matching model based on deep learning in the present invention when performing address semantic matching;
[0031] Figure 2 This is a structural diagram of the address semantic matching model based on deep learning in the present invention during training. DETAILED DESCRIPTION
[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0033] The present invention proposes an error-adaptive deep learning address semantic matching method and system, such as Figure 1 As shown, the method includes the following contents: the user inputs the non-standard address text into the trained address semantic matching model based on deep learning to obtain the standard address matching result corresponding to the non-standard address; the deep learning address semantic matching model designed by the present invention includes three layers: model layer, algorithm layer and data layer; the model layer includes BERT model (pre-trained language representation model), Bi-LSTM model (bidirectional long short-term memory network model) and Transformer model; the algorithm layer includes train algorithm and predict algorithm, wherein the train algorithm is used to train the deep learning address semantic matching model, and the predict algorithm is used to query the standard address of the non-standard address text input by the user; the data layer includes a database and a memory pool, wherein the database opens an external query interface to store the standard address data to be queried and matched, and the memory pool is used to cache the query results from the database, thereby improving the model inference matching speed by reducing the number of database IO times.
[0034] The process of processing non-standard address text using the trained deep learning-based address semantic matching model includes:
[0035] S1: Use the BERT model to process the non-standard address text to obtain the word segmentation results and word embedding vectors of the non-standard address;
[0036] For example, if the input non-standard address text is "I am at No. 1 Chongwen Road, above Nanshan, Nan'an District, Chongqing City", the BERT model processes the non-standard address text and outputs the word segmentation results and the corresponding word embedding vectors. Specifically, the data changes of the BERT model are shown in Table 1:
[0037] Table 1 Example of BERT model data changes
[0038]
[0039]
[0040] S2: The word segmentation results and word embedding vectors are simultaneously input into the Bi-LSTM model and the Transformer model to obtain the address key element information and the judgment result of whether the address key element information is available;
[0041] The Bi-LSTM is used to provide key address elements in non-standard addresses, and the Transformer is used to determine which key address elements can be used to construct database query statements. Specifically, the word segmentation results and word embedding vectors are input into the Bi-LSTM model. The Bi-LSTM model uses the word segmentation results and word embedding vectors to determine whether a word in the non-standard address text is an address key element. If so, the Bi-LSTM model determines the parent category of the address key element, obtaining address key element information, namely the address key element and its parent category. For example, the parent category of "Nan'an District" is "district level." When constructing database query statements, the training algorithm will prioritize parent categories stored in the database (such as "provincial level," "city level," and "district level") when recursively constructing query statements. The parent category of "Chongwen Road" is "road level," which is stored in an unstructured manner in the database and mixed with parent categories such as "township level," "village level," and "town level" in the same data domain. The training algorithm will try to avoid using these unstructured parent categories first when constructing query statements (relying on unstructured data will slow down query speed). The parent class of "Chongqing City" can be either "provincial level" or "municipal level," depending on whether "Chongqing City" is stored in the provincial or municipal data domain in the database. Leveraging the Bi-LSTM's RNN structure, the Bi-LSTM can learn the address structure information implicit in the input text and, with the aid of this address structure information, identify address elements. The input and output of the Bi-LSTM model are shown in Table 2:
[0042] Table 2 Example of Bi-LSTM model data changes
[0043]
[0044]
[0045] During the prediction process, the Transformer will determine whether the address key elements can be used to construct an address query statement. Specifically: the tokenized results and word embedding vectors are input into the Transformer model, and the Transformer model predicts whether the address elements are applicable to the current database. The input and output of the Transformer model are shown in Table 3. In Table 3, assume that there is a storage error for "Nan'an District" in the database, that is, "Nan'an District" is not correctly stored in the corresponding "district-level" data field. At this time, regardless of the predict algorithm or the train algorithm, if "Nan'an District" is placed in the "district-level" part of the address query statement for address query, the above query will surely not be able to correctly match the standard address. Therefore, the Transformer model will prevent data with similar storage domain errors from entering the subsequent query statement construction process. Words like "my" and "of" are not address keywords, and Bi-LSTM will also exclude similar words from the address keyword prediction results.
[0046] Table 3 Example of data changes in the Transformer model
[0047]
[0048] S3: Adopt the predict algorithm, construct a query statement according to the available address key element information for address query, and obtain the standard address matching result corresponding to the non-standard address text.
[0049] The predict algorithm constructs an address query statement according to the available address key element information, and the position of the address key elements in the address query statement is determined by the address key element parent category output by the Bi-LSTM model;
[0050] First, check whether the same address query statement exists in the memory pool. If it exists, return the standard address information queried by this address query statement to the user; otherwise, access the database according to the address query statement, query the standard address information corresponding to the query statement in the database. If the standard address is queried, return the result and construct the query statement and the query result into a key-value pair and store it in the memory pool.
[0051] If the query statement already exists in the memory pool, only one database retrieval operation needs to be performed, which can greatly reduce the database query time and the database IO pressure.
[0052] As Figure 2 shown, the process of training the address semantic matching model based on deep learning includes the following contents:
[0053] Obtain address text training data with sequence annotation labels, input the address text training data into the BERT model, and obtain the word segmentation results and word embedding vectors of the address text;
[0054] The word segmentation results and word embedding vectors are simultaneously input into the Bi-LSTM model and the Transformer model to obtain the address key element information and the discrimination results of whether the address key element information is available;
[0055] Adopting the train algorithm, the address query statement is recursively constructed based on the available address key element information to perform address query and obtain the database standard address matching result corresponding to the non-standard address text;
[0056] The model parameters are adjusted according to the standard address language matching results and labels to obtain a trained address semantic matching model based on deep learning.
[0057] The process of querying addresses using the train algorithm includes:
[0058] The train algorithm uses a recursive approach to gradually append available address key elements to the address query statement model. After each recursion, a database query is performed to continuously narrow the scope of the address query, thereby completing the matching of non-standard addresses with standard addresses. When there is only one standard address data in the query result, the standard address is the standard address matching result corresponding to the address text. Specifically:
[0059] Taking the i-th address query statement in the recursive process as an example, the train algorithm first checks the memory pool to see if the same address query statement exists. If so, it directly returns the standard address data corresponding to the i-th address query statement and the key address elements used to construct the i-th address query statement. If the address query statement does not exist in the memory pool, it is used to query the database. If the statement retrieves a single standard address, this standard address is the standard address matching result corresponding to the address text (which may be incorrect). If the latitude and longitude of the current standard address and the current non-standard address are the same, the query is successful (the matching result is correct). The train algorithm constructs a key-value pair from the address query statement and its corresponding single standard address and stores it in the memory pool. If all available address elements have been recursively searched and no standard address matching result has been found, the address matching has failed.
[0060] During the recursive process of the training algorithm, the algorithm will give priority to address elements with structured storage to construct address query statements (such as address elements with parent categories of "provincial level", "municipal level" and "district level"), thereby reducing the time consumed by database queries. When writing to the memory pool, the expiration time of the key in the memory pool can be set to ensure that the memory pool stores the hot data in the current epoch. The query results of each query statement are stored in the memory pool first; in the first cycle (epoch) of training, the training algorithm will not access the memory pool because a large amount of original data is missing from the memory pool at this time. In subsequent epochs, since the memory pool already stores a large amount of query key-value pairs, the training algorithm gives priority to accessing the memory pool. If the memory pool cache hits, the result is returned directly without querying the database, thus avoiding the additional time cost caused by frequent database IO during training.
[0061] During the training process of the address semantic matching model based on deep learning, after the train algorithm obtains the standard address matching result corresponding to the address text, it returns the address key elements used to the Transformer model and stores the query result, i.e. the standard address matching result, in the memory pool. The Transformer model corrects its own judgment result on whether the address key element information is available based on the returned address key elements, and adjusts its own parameters based on the corrected judgment result.
[0062] The present invention also provides an error-adaptive deep learning address semantic matching system, which can be used to execute the above-mentioned error-adaptive deep learning address semantic matching method, including: a BERT-based address text input module, a Bi-LSTM module, a Transformer module and an address matching module;
[0063] The BERT-based address text input module is used to perform text encoding on the address text to obtain the word segmentation result and word embedding vector of the address text;
[0064] The Bi-LSTM module is used to obtain address key element information and determine the position of address key elements in the query statement based on the word segmentation results and word embedding vectors of the address text;
[0065] The Transformer module is used to determine whether the address key element information can be used to construct an address query statement based on the word segmentation results and word embedding vectors of the address text;
[0066] The address matching module is used to construct an address query statement and perform address query based on the output information of the Bi-LSTM module and the Transformer module, and output the standard address matching result of the address text.
[0067] Compared with existing technical solutions, the present invention can narrow the gap between address semantic similarity and address matching. In existing text matching methods, models that judge whether addresses match based on text semantic similarity have difficulty identifying subtle differences between semantics, and methods that add additional address element structure information during model training lack interpretability and logic. However, the present invention deconstructs the complete semantics into finer-grained address elements, uses address elements to gradually construct address query statements, and gradually narrows the data retrieval scope, effectively reducing the model's misjudgment rate for difficult negative sample addresses. At the same time, by using the model to directly learn from the database, the probability of address matching failures due to data storage errors in the database is reduced, ensuring that the model can adaptively adjust to database errors.
[0068] Compared with the existing technical solutions, the present invention has no restrictions on the length of input data. The existing technical solutions are not only difficult to process address data with too short text length (too short text leads to the lack of some address elements, which leads to a decrease in the semantic similarity of the address), but also difficult to deal with address data with too long text length (the text is too long, and there are a lot of useless words mixed in it, which leads to a decrease in the semantic similarity of the address). The present invention uses a deep learning model to extract key address elements from the address text and complete address matching based on the address elements, so long address text will not affect address matching. At the same time, the present invention completes address matching for the database, and the database structured data is mainly used for query, and unstructured data is used as an auxiliary, so the lack of some addresses in the short address text will not affect the address matching (for example, after finding the 'city' and 'district', the 'province' first-level unit that is missing in the address can be directly obtained from the structured data in the database).
[0069] Compared with existing technical solutions, the present invention does not require time-consuming text pair construction operations. Existing technical solutions require the construction of text pairs, and no matter which construction method is used, it will consume a lot of time and energy, and the quality of text pair construction will directly affect the performance of subsequent models. At the same time, existing technical solutions pay more attention to matching accuracy and ignore the research on text pair construction methods. The above factors make the actual deployment of existing models extremely difficult. The solution proposed by the present invention is oriented towards the database to complete the address matching work. There is no need to perform text pair construction operations. Address matching can be completed by simply parsing the text to be matched, which undoubtedly has better deployability.
[0070] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. An error-adaptive deep learning address semantic matching method, characterized in that: include: The user inputs the non-standard address text into the trained address semantic matching model based on deep learning to obtain the standard address matching result corresponding to the non-standard address; The process of processing non-standard address text using the trained deep learning-based address semantic matching model includes: S1: Use the BERT model to encode the non-standard address text to obtain the word segmentation results and word embedding vectors of the non-standard address. The process of obtaining the address key element information includes: the Bi-LSTM model determines whether each word in the non-standard address text is an address key element based on the word segmentation results and word embedding vectors. If so, the parent category of the address key element is determined to obtain the address key element information, namely the address key element and its corresponding parent category. S2: The word segmentation results and word embedding vectors are simultaneously input into the Bi-LSTM model and the Transformer model to obtain the address key element information and the judgment result of whether the address key element information is available. The judgment process of whether the address key element is available includes: the Transformer model determines whether the address key element information can be used to construct the address query statement based on the current data storage status of the database, the word segmentation results and word embedding vectors output by the BERT model; S3: Use the predict algorithm to construct a query statement based on the available address key element information to perform address query and obtain the standard address matching result corresponding to the non-standard address text. The process of performing address query using the predict algorithm includes: The predict algorithm constructs an address query statement based on the available address key element information. The position of the address key element in the address query statement is determined by the parent category of the address key element output by the Bi-LSTM model. Check whether there is a standard address matching result corresponding to the address query statement in the memory pool. If so, return the standard address matching result directly. Otherwise, use the address query statement to query the database. If the standard address is found, return the result and write it to the memory pool for subsequent queries.
2. The error-adaptive deep learning address semantic matching method according to claim 1, characterized in that: The process of training the deep learning-based address semantic matching model includes: Obtain labeled address text training data and input the address text training data into the BERT model to obtain the word segmentation results and word embedding vectors of the address text; The word segmentation results and word embedding vectors are simultaneously input into the Bi-LSTM model and the Transformer model to obtain the address key element information and the discrimination results of whether the address key element information is available; Adopting the train algorithm, the address query statement is recursively constructed based on the available address key element information to perform address query and obtain the standard address matching result corresponding to the address text; The model parameters are adjusted according to the standard address matching results and labels to obtain a trained address semantic matching model based on deep learning.
3. The error-adaptive deep learning address semantic matching method according to claim 2, characterized in that: The process of constructing an address query statement using the train algorithm to perform address query includes: the train algorithm gradually appends address key elements to the address query statement through a recursive method, thereby continuously narrowing the scope of the address query; when there is only one standard address data in the query result, the standard address is the standard address matching result corresponding to the address text.
4. The error-adaptive deep learning address semantic matching method according to claim 2, characterized in that: The process of training the deep learning-based address semantic matching model also includes: after the train algorithm obtains the standard address matching result corresponding to the address text, it returns the address key elements used to the Transformer model, and at the same time stores the query result, i.e., the standard address matching result, in the memory pool. The Transformer model corrects its own judgment result on whether the address key element information is available based on the returned address key elements, and adjusts its own parameters based on the corrected judgment result.
5. An error-adaptive deep learning address semantic matching system, the system being used to execute the error-adaptive deep learning address semantic matching method according to any one of claims 1 to 4, characterized in that: include: BERT-based address text input module, Bi-LSTM module, Transformer module, and address matching module; The BERT-based address text input module is used to perform text encoding on the address text to obtain the word segmentation result and word embedding vector of the address text; The Bi-LSTM module is used to obtain address key element information and determine the position of address key elements in the query statement based on the word segmentation results and word embedding vectors of the address text; The Transformer module is used to determine whether the address key element information can be used to construct an address query statement based on the word segmentation results and word embedding vectors of the address text; The address matching module is used to construct an address query statement and perform address query based on the output information of the Bi-LSTM module and the Transformer module, and output the standard address matching result of the address text.