Address data repair method

Abstract

The application discloses an address data repairing method, comprising: obtaining and enhancing an address data set to obtain a training data pair; initializing an address data repairing model, the address data repairing model comprising a preset encoder based on a Transformer and a preset decoder based on a bidirectional long short-term memory network, the preset encoder performing fusion level position coding operation and dependency relationship modeling based on a multi-head self-attention mechanism on an input address sequence to obtain a hidden state sequence, and the preset decoder performing decoding operation on the hidden state sequence to obtain a repaired address sequence; training the address data repairing model according to the training data pair; inputting a current address to be repaired into the trained address data repairing model to obtain a target repaired address. The application encodes the fusion level position in combination with the multi-head self-attention mechanism, can better utilize address level information, and thus improves the repairing accuracy of the address to be repaired.

Description

Technical Field

[0001] This application relates to the field of data restoration technology, and in particular to a method for restoring address data. Background Technology

[0002] With the rapid development of e-commerce, logistics, and smart cities, address data, as crucial foundational information, directly impacts operational efficiency and service quality. However, in practical applications, due to differences in user input habits, limitations in data collection environments, or improper storage of historical records, address data commonly suffers from low-quality issues such as missing information, incorrect descriptions, and non-standard formats, significantly hindering subsequent address resolution, spatial positioning, and data analysis. Therefore, effectively repairing and enhancing low-quality address data has become an urgent problem to be solved in related technical fields.

[0003] Existing methods for processing low-quality address data mainly include rule-based methods, statistical machine learning methods, and deep learning-based methods. While deep learning-based methods can automatically learn data features through neural networks, existing solutions often employ a general sequence-to-sequence model architecture, failing to fully consider the inherent hierarchical structure of address data. Specifically, address data consists of multiple levels, such as province, city, district, street, and detailed address, with clear hierarchical relationships and semantic constraints between levels. Traditional Transformer models, when processing this type of data, can only express absolute positional information within the sequence through their positional encoding, failing to distinguish the differences in identity between elements at different levels within the address structure. This makes it difficult for the model to accurately capture cross-level dependencies, leading to hierarchical errors and mismatches between upper and lower levels in the repair results, thus limiting the accuracy and robustness of address repair. Summary of the Invention

[0004] The main objective of this application is to provide an address data repair method that addresses the technical problem that existing deep learning-based address data character methods do not take into account the hierarchical structure of address data, thus leading to a lack of accuracy in address repair results.

[0005] To achieve the above objectives, this application proposes an address data repair method, the method comprising: Obtain an address dataset and perform augmentation processing on the address dataset to obtain training data pairs, wherein the training data pairs include address samples to be repaired and corresponding repaired address samples; An address data repair model is initialized. The address data repair model includes a preset encoder based on Transformer and a preset decoder based on bidirectional long short-term memory network. The preset encoder is used to perform encoding operations on the fusion level position of the input address sequence and dependency modeling based on multi-head self-attention mechanism to obtain the hidden state sequence. The preset decoder is used to perform decoding operations on the hidden state sequence to obtain the repaired address sequence. The address data repair model is trained based on the training data to obtain the trained address data repair model; The current address to be repaired is input into the trained address data repair model to obtain the target repaired address.

[0006] In one embodiment, the step of obtaining an address dataset and performing augmentation processing on the address dataset to obtain training data pairs includes: Obtain the address dataset and parse each address in the address dataset into a sequence of hierarchical fields; A random masking operation is performed on the hierarchical field sequence, and the selected field is replaced with a mask character according to a preset probability; Perform a redundant word removal operation on the masked hierarchical field sequence to identify and remove redundant words in the masked hierarchical field sequence; Perform an error element replacement operation on the hierarchical field sequence after deleting redundant words, replacing the selected field with a preset error version; The hierarchical field sequence after field replacement is used as the address sample to be repaired, and the hierarchical field sequence is used as the repaired address sample to construct training data pairs.

[0007] In one embodiment, the step of training the address data repair model based on the training data to obtain the trained address data repair model includes: The training data is divided into several training batches to obtain the sample input sequence and sample output sequence corresponding to each required training batch. The sample input sequence corresponding to the current batch is input into the preset encoder to obtain the hidden state sequence corresponding to the current batch. The hidden state sequence corresponding to the current batch is input into the preset decoder to obtain the repaired address sequence corresponding to the current batch; The cross-entropy loss function is calculated based on the repaired address sequence and the sample output sequence corresponding to the current batch, and the parameters of the address data repair model are updated based on the calculation results. Then, the process returns to the step of inputting the sample input sequence corresponding to the current batch into the preset encoder. The training of the address data repair model is completed when the current batch meets the preset convergence condition.

[0008] In one embodiment, the preset encoder includes: an embedding layer, an encoder layer, and an encoder output layer; The step of inputting the sample input sequence corresponding to the current batch into the preset encoder to obtain the hidden state sequence corresponding to the current batch includes: The embedding layer performs hierarchical position encoding on each element in the sample input sequence, and combines the hierarchical position encoding of each element with the word embedding of each element to obtain the input embedding of each element and construct the input matrix. The encoder layer obtains an attention output matrix based on the multi-head self-attention mechanism and the input matrix, and performs element-level filtering on the attention output matrix based on a gating mechanism. The hidden state sequence is obtained through the encoder output layer based on the filtering results.

[0009] In one embodiment, the step of performing hierarchical position encoding on each element in the sample input sequence through the embedding layer, and combining the hierarchical position encoding of each element with the word embedding of each element to obtain the input embedding of each element and constructing an input matrix includes: Determine the hierarchical index corresponding to each element in the sample input sequence. The hierarchical index is used to characterize the belonging level of each element in the preset standard address hierarchy structure. Calculate the standard sinusoidal position code corresponding to each element, and combine the standard sinusoidal position code with the level-specific embedding corresponding to the level index to obtain the level position code corresponding to each element. The level-specific embedding is the learnable parameter matrix and bias vector corresponding to each level index. The word embedding of each element is determined based on the sample input sequence, and the hierarchical position code corresponding to each element is added to the word embedding of each element to obtain the input embedding of each element; The input embeddings of each element are integrated to obtain the input matrix.

[0010] In one embodiment, the step of obtaining the attention output matrix through the encoder layer based on the multi-head self-attention mechanism and the input matrix includes: The query matrix, key matrix, and value matrix are calculated based on the shared weight matrix and the input matrix. Based on the multi-head self-attention mechanism, the query matrix, key matrix, and value matrix corresponding to each attention head are determined. The head output features of each attention head are determined based on the query matrix, key matrix, and value matrix corresponding to each attention head. The head output features of each attention head are spliced ​​together, and an attention output matrix is ​​obtained based on the splicing result and a preset output weight matrix.

[0011] In one embodiment, the step of inputting the hidden state sequence corresponding to the current batch into the preset decoder to obtain the repaired address sequence corresponding to the current batch includes: The hidden state at the last position in the hidden state sequence is used as the initial hidden state of the decoder; At each time step, the attention weights of each decoding position are determined based on the soft attention mechanism and weighted to obtain the context vector. Combining the context vector and the initial embedding vector, a forced teaching strategy is adopted to obtain the current hidden state of the decoder and determine the embedding vector corresponding to each time step. The embedding vector corresponding to the last time step is used as the repaired address sequence corresponding to the current batch.

[0012] In one embodiment, the preset decoder includes a bidirectional long short-term memory network layer; The step of determining the attention weights for each decoding position at each time step based on a soft attention mechanism and weighting them to obtain a context vector, and then combining the context vector with the initial embedding vector and employing a forced teaching strategy to obtain the current hidden state of the decoder to determine the embedding vector corresponding to each time step, includes: At the current time step, based on the hidden state of the preset decoder in the previous time step and the hidden state sequence, an attention score is calculated, and an attention weight is obtained according to the attention score. The hidden state of the preset decoder in the previous time step is obtained by updating the initial hidden state of the decoder. The hidden states at each position in the hidden state sequence are weighted and summed according to the attention weights to obtain the context vector of the current time step; The embedding vector corresponding to the previous time step is concatenated with the context vector of the current time step and used as the decoder input of the current time step. The embedding vector output by the previous time step is the initial embedding vector or the real embedding vector determined by the forced teaching strategy based on the repaired address sequence. The decoder input and the hidden state of the preset decoder at the previous time step are input into the bidirectional long short-term memory network layer to obtain the current hidden state of the encoder; The current hidden state is processed by linear transformation and normalized exponential function to output the embedding vector corresponding to the current time step.

[0013] In one embodiment, the step of inputting the current address to be repaired into the trained address data repair model to obtain the target repaired address includes: Input the current address to be repaired into the trained address data repair model; The current address to be repaired is encoded by the preset encoder that has been trained to obtain the current hidden state sequence; The current hidden state sequence is decoded by the preset decoder that has been trained, and the decoding result is sampled using a beam search algorithm to obtain several candidate repaired address sequences. Each candidate repaired address sequence is filtered and scored using address validity rules, and the candidate repaired address sequence with the highest score is selected as the target repaired address.

[0014] In one embodiment, after the step of applying address validity rules to each of the candidate repaired address sequences for filtering and scoring, and selecting the candidate repaired address sequence with the highest score as the target repaired address, the method further includes: When the target repaired address passes verification, the current address to be repaired and the target repaired address are constructed into a correct repair data pair; The correctly repaired data is incorporated into the training data pair to obtain the updated training data pair; The updated training data is used to repair the model based on the optimized address data.

[0015] This application discloses an address data restoration method, comprising: acquiring an address dataset and performing augmentation processing on the address dataset to obtain training data pairs, wherein the training data pairs include address samples to be restored and corresponding restored address samples; initializing an address data restoration model, wherein the address data restoration model includes a preset encoder based on Transformer and a preset decoder based on bidirectional long short-term memory network, wherein the preset encoder is used to perform encoding operations on the fusion level positions of the input address sequence and dependency modeling based on multi-head self-attention mechanism to obtain a hidden state sequence, and the preset decoder is used to perform decoding operations on the hidden state sequence to obtain a restored address sequence; training the address data restoration model according to the training data to obtain a trained address data restoration model; and inputting the current address to be restored into the trained address data restoration model to obtain the target restored address.

[0016] Because this application integrates hierarchical positional encoding into the encoder, the model can distinguish the identities of elements at different levels, avoiding the limitation of traditional positional encoding that only expresses the absolute position of the sequence, thus more accurately capturing the hierarchical relationships between address elements. Simultaneously, by modeling dependencies through a multi-head self-attention mechanism and combining it with a bidirectional long short-term memory network decoder to decode the hidden state sequence, the model can take contextual information into account during the repair process, improving the accuracy of repairing missing, erroneous, or redundant address elements. Furthermore, by augmenting the address dataset to generate training data pairs, the model can learn multiple types of address error patterns, further enhancing its generalization ability and robustness. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the address data repair method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the address data repair method of this application; Figure 3 A schematic diagram of the module architecture for the pre-defined encoder in this application; Figure 4 This is a flowchart illustrating the third embodiment of the address data repair method of this application; Figure 5 This is a schematic diagram of the module architecture of the pre-defined decoder in this application; Figure 6 This is a flowchart illustrating the fourth embodiment of the address data repair method of this application; Figure 7 This is a schematic diagram illustrating the entire process of the address data repair method in this application. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] This application provides an address data repair method, referencing... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the address data repair method of this application. In this embodiment, the method includes: steps S10~S40: Step S10: Obtain the address dataset and perform augmentation processing on the address dataset to obtain training data pairs, wherein the training data pairs include address samples to be repaired and corresponding repaired address samples.

[0023] It should be noted that the method of this embodiment can be applied to scenarios requiring the repair of low-quality address data, such as logistics and distribution, e-commerce systems, and urban planning and management. The executing entity of this embodiment can be a computing electronic device with functions such as data processing, model training, and network communication, such as a server, desktop computer, or cloud computing platform. Here, an address data repair device (referred to as "device") is used as an example to illustrate this embodiment and the following embodiments.

[0024] It should be understood that address datasets can be high-quality collections of address data gathered from business systems, such as delivery addresses successfully signed for by users on e-commerce platforms or delivery addresses accurately delivered in logistics systems. This address data is complete and standardized, and can serve as raw material for subsequent enhancement processing.

[0025] After acquiring the address dataset, the device can perform enhancement processing on each address in the dataset. By simulating address quality issues that may occur in real-world applications, it generates pairs of address samples to be repaired and repaired address samples for model training. This enhancement processing includes operations such as randomly masking the hierarchical field in the address, removing redundant modifiers, and replacing some erroneous elements, thereby constructing training data pairs that cover multiple error types.

[0026] Furthermore, to illustrate in detail how to obtain training data pairs, including the address sample to be repaired and the corresponding repaired address sample, from a high-quality address dataset, step S10 specifically includes: steps S101~S105: Step S101: Obtain the address dataset and parse each address in the address dataset into a sequence of hierarchical fields.

[0027] It should be noted that address data has an inherent hierarchical structure, typically including multiple levels such as province, city, district, street, and detailed address. The device can parse each address into a sequence of fields according to its hierarchical structure, for example... ,in Indicates the first Each field (such as province, city) This is the total number of fields.

[0028] Step S102: Perform a random masking operation on the hierarchical field sequence, and replace the selected field with a mask character according to a preset probability.

[0029] It should be noted that the random masking operation is used to simulate the situation where fields are missing in the address. The device can independently decide whether to replace each field in the hierarchical field sequence with a mask symbol according to a preset mask probability. This mask symbol is a special marker used to indicate that field information is missing at the current location.

[0030] It should be understood that, firstly, a mask address can be generated with probability. Random selection field Replace with mask character Specifically, this is achieved through uniform distribution sampling, as shown in the formula:

[0031] in, , It is the mask probability parameter (usually set to 0.1-0.3). It is an independent random variable.

[0032] In a specific implementation, the device can generate a random number sequence with the same length as the hierarchical field sequence. For each field, if the corresponding random number is less than the mask probability, the field is replaced with a mask character; otherwise, the original field is retained.

[0033] Step S103: Perform a redundant word removal operation on the masked hierarchical field sequence to identify and remove redundant words in the masked hierarchical field sequence.

[0034] It should be noted that the redundant word removal operation can be used to simulate situations where the address contains redundant modifiers. The device can predefine a set of redundant words, which contains common redundant modifiers in address fields, such as suffixes like "province", "city", "district", "street", and "road".

[0035] It should be understood that a set of redundant words can be defined:

[0036] For each field If it contains Then, based on probability The formula to remove this word is:

[0037] in, , This is the deletion probability (recommended value: 0.05-0.2). The function removes redundant suffixes from the matched list.

[0038] In a specific implementation, the device can traverse each field in the sequence of hierarchical fields after the mask, and if a field contains a word in the redundant word set, it decides whether to remove the redundant word from the field according to a preset deletion probability. The removal operation only deletes the redundant word part and retains the core information in the field.

[0039] Step S104: Perform an error element replacement operation on the sequence of hierarchical fields after removing redundant words, and replace the selected field with a preset error version.

[0040] It should be noted that the error element replacement operation can be used to simulate the situation where there are typos or incorrect expressions in an address. The device can pre-define an error mapping set, which contains the correspondence between correct words and common error versions, such as "Beijing" corresponding to "Beijiu", "Shanghai" corresponding to "Shanghai", "Guangzhou" corresponding to "Guangzhou", etc.

[0041] It should be understood that the device can define an error mapping set:

[0042] Furthermore, with a probability for each field sample an error version, and the formula is:

[0043] where , is the replacement probability (typical value 0.05 - 0.15), the function randomly selects an error item corresponding to from .

[0044] In a specific implementation, the device can traverse each field in the sequence of hierarchical fields after removing redundant words. If the field content exists in the keys of the error mapping set, it decides whether to replace the field with the corresponding error version according to a preset replacement probability.

[0045] Step S105: Use the sequence of hierarchical fields after field replacement as the sample of the address to be repaired, and use the sequence of hierarchical fields as the sample of the repaired address to construct a training data pair.

[0046] It should be noted that after the above-mentioned random masking, redundant word deletion, and error element replacement operations, the original high-quality address is converted into a low-quality address with missing information, redundant modification, or incorrect expression. This low-quality address can be used as a sample of the address to be repaired, and the original untreated high-quality address can be used as the corresponding sample of the repaired address. The two together form a training data pair.

[0047] Specifically, the address sample to be repaired can be represented as: The corresponding repaired address sample is then... These together constitute the training data pairs. .

[0048] It should also be noted that, through the above data augmentation processing, the device can generate a large number of training data pairs covering various error types, providing rich sample data for subsequent model training. Furthermore, the parameters in the above data augmentation processing... It can be optimized through cross-validation based on the dataset distribution, ensuring enhanced diversity without compromising semantic structure.

[0049] Step S20: Initialize the address data repair model, which includes a preset encoder based on Transformer and a preset decoder based on bidirectional long short-term memory network. The preset encoder is used to perform encoding operations on the fusion level position of the input address sequence and dependency modeling based on multi-head self-attention mechanism to obtain the hidden state sequence. The preset decoder is used to perform decoding operations on the hidden state sequence to obtain the repaired address sequence.

[0050] It should be noted that the address data restoration model can adopt an encoder-decoder architecture. The preset encoder is based on Transformer and improved from the original Transformer, including: designing a hierarchical position encoding mechanism for the hierarchical structure of addresses (province, city, district, street level); using a multi-head self-attention mechanism to capture the dependencies between elements at different levels; and adding a gating mechanism to filter irrelevant information, thereby outputting a hidden state sequence.

[0051] The preset decoder can be built on a bidirectional long short-term memory (BiLSTM) network. The preset decoder can use BiLSTM layers to process the hidden state sequence output by the encoder, and combine a soft attention mechanism to gradually generate the repaired address sequence from the hidden state sequence output by the encoder.

[0052] In a specific implementation, the device can initialize various parameters of the preset encoder and preset decoder, including: word embedding matrix, learnable parameters of hierarchical position encoding, multi-head attention weight matrix, gating mechanism parameters, BiLSTM network parameters, and output layer parameters.

[0053] The preset encoder receives the input address sequence, performs hierarchical positional encoding fusion, multi-head self-attention calculation, and gating filtering, and outputs a hidden state sequence. The preset decoder uses the last hidden state output by the preset encoder as its initial state. At each time step of decoding, it dynamically focuses on the address elements in the encoder related to the current decoding position through a soft attention mechanism, and updates the hidden state based on BiLSTM to gradually generate the repaired address sequence.

[0054] Step S30: Train the address data repair model based on the training data to obtain the trained address data repair model.

[0055] It should be noted that the device can use the training data generated in the aforementioned steps to train the address data repair model. By iteratively optimizing the model parameters, the model can accurately repair high-quality address outputs from low-quality address inputs.

[0056] Furthermore, to illustrate in detail how to train the address data repair model based on the training data, step S30 specifically includes: steps S301~S305: Step S301: Divide the training data pair into several training batches to obtain the sample input sequence and sample output sequence corresponding to each required training batch.

[0057] It should be noted that, to improve training efficiency, the device can divide all training data pairs into multiple mini-batches, each batch being B in size and containing a fixed number of training samples. The input sequence for each training sample is the address sample to be repaired, and the output sequence is the corresponding repaired address sample.

[0058] It should also be noted that, for the initialized address data repair model, its encoder parameters can be expressed as... The decoder parameters are expressed as Set the final output layer weight matrix of the model. and bias Furthermore, the model parameters can be initialized using Xavier, and the Adam optimizer can be selected for subsequent iterations, along with the learning rate. Set to 0.001.

[0059] Step S302: Input the sample input sequence corresponding to the current batch into the preset encoder to obtain the hidden state sequence corresponding to the current batch.

[0060] It should be noted that, for the current training batch, the device can input the corresponding sample input sequence into the preset encoder. The preset encoder first encodes the embedding of each input token in the sample input sequence, then calculates the dependencies between elements at different levels through a multi-head self-attention mechanism, and then filters irrelevant information through a gating mechanism, finally outputting the hidden state corresponding to each input position, forming a hidden state sequence.

[0061] It should be understood that, for each training batch, the sample input sequence is represented as... ,in It is a token embedding vector. is the embedding dimension, and n is the length of the input sequence.

[0062] The preset encoder processes the sample input sequence and calculates the hidden state sequence: initial hidden state. For each time step (From 1 to n), the corresponding embedding vector of each token can be read sequentially, and the value is updated after each one is read. The final context vector This refers to the hidden state at the last time step, which serves as a summary representation of the entire sequence, containing all input information. , This represents the size of the hidden layer.

[0063] Step S303: Input the hidden state sequence corresponding to the current batch into the preset decoder to obtain the repaired address sequence corresponding to the current batch.

[0064] It should be noted that the device can input the hidden state sequence output by the preset encoder into the preset decoder. The preset decoder initializes its hidden state with the last hidden state of the preset encoder's output sequence. At each time step of decoding, it calculates the attention weights between the current decoding position and each hidden state of the encoder using a soft attention mechanism, generating a context vector. Then, it concatenates the previous step's output with the context vector and inputs it into the BiLSTM layer to update the hidden state. Finally, it outputs the address token for the current time step through a linear transformation and a softmax function. The preset decoder can repeat the above process until an end symbol is generated, resulting in the complete repaired address sequence.

[0065] It should be understood that the preset decoder gradually generates the repaired address sequence, i.e., the repaired address token sequence, based on the hidden state sequence output by the aforementioned encoder. Firstly, a special starting token can be set. Initial decoder state This refers to the final output of the aforementioned preset encoder. For each time step t (t=1 to T, where T is the length of the sample output sequence), the previous real token is input. (the first of the sample output sequences) (each token), the hidden state of the previous time step Calculate the current hidden state:

[0066] Next, a linear transformation and softmax are applied to the output layer to obtain the probability distribution:

[0067] in, , For vocabulary size, To obtain a vector of dimension V through linear transformation, softmax transforms the vector into a probability distribution, where each probability value represents a probability distribution for the token. The probability of.

[0068] Step S304: Calculate the cross-entropy loss function based on the repaired address sequence and the sample output sequence corresponding to the current batch, update the parameters of the address data repair model based on the calculation results, and return to the step of inputting the sample input sequence corresponding to the current batch into the preset encoder.

[0069] It should be noted that the device can compare the repaired address sequence output by the preset decoder with the sample output sequence (i.e., the real repaired address) corresponding to the current batch and calculate the cross-entropy loss function.

[0070] It should be understood that this loss function can be used to measure the difference between the model's predictions and the actual results. Based on the calculated loss value, the device calculates the gradients of each parameter in the model using the backpropagation algorithm, and updates the parameters of the encoder and decoder using an optimizer (such as the Adam optimizer). After completing the parameter update for the current batch, it returns to the aforementioned step S302 to continue processing the next training batch.

[0071] It should also be noted that the sample output sequence corresponding to the current batch is... ,in It is the first The cross-entropy loss function can be expressed as follows: (The original text contains several grammatical errors and inconsistencies. A more accurate translation would require the full context.)

[0072] In the formula, For the model at time step For the probability value of the correct token, if the model predicts a high probability (close to 1), then -log(1) = 0, and the loss value is small; if the predicted probability is low (close to 0), then -log(0) = The loss value is large. The total loss obtained by summing over all time steps (T) can reflect the prediction accuracy of the model over the entire sequence.

[0073] Next, the device can perform backpropagation to update the model parameters and calculate the loss. Relative to all trainable parameters gradient The gradient can indicate in which direction each number should be adjusted to minimize the loss.

[0074] Then, using the chain rule and automatic differentiation, gradient updates are applied through the optimizer Adam:

[0075] The Adam optimizer can adaptively adjust the learning rate based on first-order and second-order moment estimates.

[0076] Step S305: Obtain the trained address data repair model when the current batch meets the preset convergence condition.

[0077] It should be noted that the device can repeatedly execute the above batch training process until the model reaches the preset convergence condition. The convergence condition may include the validation set loss changing less than a preset threshold for multiple consecutive training epochs, or reaching a preset maximum number of training epochs. When the convergence condition is met, the device stops training, saves the current model parameters, and obtains the address data of the completed training to repair the model.

[0078] For example, the device can loop through the entire training process on other batches, completing one epoch by traversing all batches, while simultaneously monitoring the loss for each batch and each epoch. When the loss change over five consecutive epochs is less than a threshold... Alternatively, the process can stop when the maximum number of epochs (e.g., 100) is reached, which is considered convergence. This confirms that training is complete and saves the model parameters, thereby obtaining the address data of the completed training to repair the model.

[0079] Step S40: Input the current address to be repaired into the trained address data repair model to obtain the target repaired address.

[0080] It should be noted that after the model training is complete, the device can use low-quality addresses that need to be repaired in real-world applications as the current addresses to be repaired, and input them into the trained address data repair model. The model first encodes the addresses to be repaired through an encoder to obtain a hidden state sequence, and then gradually generates a repaired address token sequence through a decoder, finally outputting the target repaired address.

[0081] This embodiment generates training data pairs covering multiple error types through data augmentation, providing the model with rich learning samples. By constructing a Transformer encoder that integrates hierarchical positional encoding, the model can accurately capture the hierarchical relationships of address elements. By constructing a BiLSTM decoder and combining it with a soft attention mechanism, the model can dynamically focus on key information in the input address during the repair process. Through end-to-end training, the model learns the mapping relationship from low-quality addresses to high-quality addresses. Compared to traditional rule-based or statistical machine learning methods, this embodiment can automatically learn the features and patterns of address data without manually formulating complex rules. It can better handle long-distance dependencies and the hierarchical structure of addresses, effectively improving the accuracy and efficiency of repairing low-quality address data.

[0082] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the address data repair method of this application.

[0083] In this embodiment, to specifically illustrate how the input sequence is encoded using a preset encoder built based on the improved Transformer, the preset encoder can be further divided into an embedding layer, an encoder layer, and an encoder output layer. Therefore, step S302 specifically includes: steps S3021~S3023: Step S3021: Perform hierarchical position encoding on each element in the sample input sequence through the embedding layer, and combine the hierarchical position encoding of each element with the word embedding of each element to obtain the input embedding of each element and construct the input matrix.

[0084] It should be noted that this can be used as a reference. Figure 3 The encoding operation process of the preset encoder is explained. Figure 3 This is a schematic diagram of the module architecture for the pre-defined encoder in this application. Figure 3It can be seen that in this preset encoder, the embedding layer can sequentially perform the functions of word segmentation and tokenization, word embedding lookup, hierarchical position encoding and embedding fusion, thereby converting the discrete tokens in the sample input sequence into a continuous vector representation, while incorporating the hierarchical structure information of the address.

[0085] Further, step S3021 includes steps S30211 to S30214: Step S30211: Determine the hierarchical index corresponding to each element in the sample input sequence. The hierarchical index is used to characterize the belonging level of each element in the preset standard address hierarchy structure.

[0086] It should be noted that address data has an inherent hierarchical structure, typically including multiple levels such as province, city, district, street, and detailed address. The device can predefine hierarchical index mapping rules, for example, mapping the province level to index 1, the city level to index 2, the district level to index 3, the street level to index 4, and the detailed address level to index 5. For each element in the input sequence, the device determines the corresponding hierarchical index based on its hierarchical position in the address.

[0087] It should be understood that hierarchical indexes can be defined. These correspond to the provincial, municipal, district, and street levels, as well as the detailed key node level. The detailed key node level can aggregate fine-grained elements such as roads, residential areas, natural villages, and points of interest (POIs) to help determine the first four levels of administrative divisions. Indicates the position in the sequence.

[0088] Step S30212: Calculate the standard sinusoidal position code corresponding to each element, and combine the standard sinusoidal position code with the level-specific embedding corresponding to the level index to obtain the level position code corresponding to each element. The level-specific embedding is the learnable parameter matrix and bias vector corresponding to each level index.

[0089] It should be noted that the standard sine position encoding The formula for representing the absolute position information of an element in a sequence is as follows:

[0090] and

[0091] in, It is the size of the embedding dimension. It is a dimensional index ( ), It represents the absolute position of an element in the sequence.

[0092] Next, the device can maintain a set of learnable parameter matrices and bias vectors for each level index to generate level-specific embeddings.

[0093] Finally, for the i-th element, its hierarchical position is encoded. This can be achieved by encoding the standard sine wave position. With the learnable parameter matrix of the corresponding level Multiply by, and then by the learnable bias vector of the corresponding level. Adding them together, we get:

[0094] In the formula, This represents the hierarchical index of the i-th element. and These are the learnable parameter matrix and bias vector corresponding to this level, respectively.

[0095] Step S30213: Determine the word embedding of each element according to the sample input sequence, and add the hierarchical position code corresponding to each element to the word embedding of each element to obtain the input embedding of each element.

[0096] It should be noted that word embeddings can be used to map discrete tokens into continuous vector representations, capturing the semantic information of the tokens themselves. The device uses a predefined word embedding matrix to convert each token in the input sequence into its corresponding word embedding vector. For the i-th element, its input embedding... By embedding words With hierarchical location encoding Adding them together, we get:

[0097] in, This represents the word embedding vector of the i-th token. This represents the hierarchical position code of the i-th element.

[0098] Through the above addition operation, the input embedding of each element is... It contains both semantic information and hierarchical structure information.

[0099] Step S30214: Integrate the input embeddings of each element to obtain the input matrix. It should be noted that the device can embed the input of all elements in the input sequence. Stacked sequentially, they form the input matrix E, which has a dimension of n. d, where n is the length of the input sequence and d is the embedding dimension. The input matrix E can be used as input to the encoder layer and participate in subsequent multi-head self-attention calculations.

[0100] Step S3022: Obtain the attention output matrix through the encoder layer based on the multi-head self-attention mechanism and the input matrix, and perform element-level filtering on the attention output matrix based on the gating mechanism.

[0101] It should be understood that, by Figure 3 It is known that the preset encoder can include multiple encoder layers. Each single-layer encoder can capture the dependencies between different elements in the input sequence through a multi-head self-attention mechanism and filter redundant information unrelated to address repair through a gating mechanism.

[0102] Furthermore, to illustrate in detail how to capture the dependencies between elements at different levels based on the multi-head self-attention mechanism, please refer to the above. Figure 3 The four functions implemented by multi-head self-attention (linear projection, multi-head segmentation, attention calculation, and multi-head merging) are further broken down into steps S30221~S30224 corresponding to the four functions: Step S30221: Calculate the query matrix, key matrix, and value matrix based on the shared weight matrix and the input matrix.

[0103] It should be noted that the device first calculates the query matrix Q, key matrix K, and value matrix V by multiplying the input matrix by three shared weight matrices. The specific calculation formula is as follows:

[0104]

[0105]

[0106] In the formula, , , The learnable shared weight matrix has dimensions of 1. , For each dimension of attention head, h represents the number of heads, i.e., the number of attention heads. The dimensions of Q, K, and V are all... , where n is the length of the input sequence.

[0107] Step S30222: Based on the multi-head self-attention mechanism, determine the query matrix, key matrix, and value matrix corresponding to each attention head.

[0108] It should be noted that the multi-head self-attention mechanism can decompose the attention computation into multiple parallel attention heads, each of which uses an independent projection matrix to transform the query, key, and value.

[0109] For the j-th head, its corresponding query matrix Key matrix Sum matrix This is obtained by multiplying Q, K, and V by the specific weight matrix of the head, and is expressed as:

[0110]

[0111]

[0112] in, The learnable weight matrix specific to the j-th head, with dimensions 1. .

[0113] Step S30223: Determine the head output features of each attention head based on the query matrix, key matrix, and value matrix corresponding to each attention head.

[0114] It should be noted that each attention head independently calculates its attention weights, and then performs a weighted summation of the value matrix accordingly. Therefore, for the j-th head, the formula for calculating its output features is:

[0115] In the formula, The attention score matrix is ​​calculated and divided by Scaling is applied to prevent gradient saturation; the softmax function normalizes the attention scores to a probability distribution; and finally, it is combined with... Multiplying yields the weighted output. Each The dimension is .

[0116] Step S30224: Concatenate the head output features of each attention head, and obtain the attention output matrix based on the concatenation result and the preset output weight matrix.

[0117] It should be noted that the device concatenates the output features of each attention head along the feature dimension, resulting in a dimension of The concatenated matrix. Because... equal The dimensions of the concatenated matrix are restored to 1. .

[0118] Next, the device multiplies the splicing matrix by a preset output weight matrix. The final attention output matrix A is obtained:

[0119] In the formula, The output weight matrix is ​​a learnable matrix. ,and = .

[0120] It should also be noted that after completing the multi-head self-attention calculation, the device can further perform element-level filtering of the attention output matrix through a gating mechanism. Specifically, the device calculates the gating signal G:

[0121] in, It is the sigmoid activation function. and These are learnable parameters. Each element of the gating signal G takes a value between 0 and 1, representing the degree to which the corresponding positional information is preserved.

[0122] The device performs element-wise multiplication between the gating signal and the attention output matrix to obtain the filtered output:

[0123] in, This represents element-wise multiplication to suppress noise and preserve key hierarchical dependencies.

[0124] It should be understood that, through the gating mechanism, the model can automatically suppress redundant information and retain key hierarchical dependencies related to address repair.

[0125] Step S3023: Obtain the hidden state sequence through the encoder output layer based on the filtering results.

[0126] It should be noted that the encoder output layer can further process the gated output, including layer normalization and feedforward network transformation, to enhance the model's expressive power and training stability.

[0127] Specifically, the device first performs layer normalization on the filtered output, adjusting the numerical distribution of each dimension to a standard range with a mean of 0 and a variance of 1, thereby accelerating model convergence. Next, the device inputs the normalized result into a feedforward network, enhancing the feature representation capability through nonlinear transformation. This feedforward network typically consists of two linear transformation layers and a nonlinear activation function. Finally, as... Figure 3 As can be seen from the encoder output layer, the device can output the hidden state sequence processed by the encoder. Its dimensions are This is then connected to a subsequent preset decoder as input to the preset decoder.

[0128] In this embodiment, the preset encoder adopts an improved Transformer-based architecture, whose overall structure includes an embedding layer, an encoder layer, and an encoder output layer. The embedding layer performs hierarchical position encoding on each element in the input sequence and merges the hierarchical position encoding with word embeddings to generate the input embedding. The encoder layer captures the dependencies between elements at different levels using a multi-head self-attention mechanism and filters out irrelevant information using a gating mechanism. The encoder output layer normalizes the output of the encoder layer to obtain the final hidden state sequence. Based on this encoder structure, the device can perform deep encoding on low-quality input addresses, explicitly incorporating the hierarchical structure information of the addresses during the encoding process. It captures the dependencies between elements at different levels through a multi-head self-attention mechanism and filters out redundant information using a gating mechanism, thereby obtaining a more expressive hidden state representation and providing a foundation for accurate decoding.

[0129] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the address data repair method of this application.

[0130] In this embodiment, to specifically illustrate how to decode the hidden state sequence output by the preset encoder based on the preset decoder constructed using BiLSTM, step S303 includes: steps S3031~S3033: Step S3031: Use the hidden state at the last position in the hidden state sequence as the initial hidden state of the decoder.

[0131] It should be noted that you can also refer to this section. Figure 5 The decoding process of the preset decoder is explained, such as... Figure 5 As shown, Figure 5 This is a schematic diagram of the module architecture for the pre-defined decoder in this application. Figure 5 It can be seen that in this preset decoder, when the output of the aforementioned encoder is received, the decoder can first be initialized, that is, the decoder hidden state is initialized to the output of the last layer of the encoder.

[0132] Specifically, the hidden state sequence output by the encoder It contains n hidden states, represented as each The hidden state corresponding to the input address element. The device can transmit the hidden state sequence. The hidden state of the last position (i.e., the nth position) Extracted and used as the initial hidden state for the decoder. This ensures that the decoder inherits the global sequence features captured by the encoder when it starts generating, providing a foundation for the subsequent decoding process.

[0133] Step S3032: In each time step, the attention weights of each decoding position are determined based on the soft attention mechanism and the weighted average is used to obtain the context vector. Combining the context vector and the initial embedding vector, a forced teaching strategy is adopted to obtain the current hidden state of the decoder and determine the embedding vector corresponding to each time step.

[0134] It should be noted that the decoder outputs the repaired address sequence using a step-by-step generation method, generating an embedding vector corresponding to an address token at each time step. Figure 5 It can be seen that the preset decoder includes a bidirectional long short-term memory network layer, namely the BiLSTM layer.

[0135] Furthermore, in order to take a single time step as an example, for Figure 5 The state update process implemented by BiLSTM is described in step S3032, which also includes steps S30321 to S30325: Step S30321: At the current time step, based on the hidden state of the previous time step of the preset decoder and the hidden state sequence, calculate the attention score and obtain the attention weight according to the attention score. The hidden state of the previous time step of the preset decoder is obtained by updating the initial hidden state of the decoder.

[0136] It should be noted that the device calculates soft attention weights at each decoding time step to determine which positions in the encoder hidden state sequence are most important for the current decoding step.

[0137] For the t-th time step, the device bases its data on the hidden state of the decoder from the previous time step. With encoder hidden state sequence Each hidden state Calculate attention score :

[0138] In the formula, It is the hidden state of the decoder in the previous step. It is the encoder number The hidden state of an element. , , , These are the learning parameters, where tanh is the hyperbolic tangent activation function. This represents the dimension of the attention hidden layer. The attention score indicates... and The degree of matching between the two positions; the higher the score, the more important that position is to the current decoding step.

[0139] After obtaining the attention score, the device normalizes it using the softmax function to obtain the attention weights. :

[0140] It should be understood that attention weights The value of is between 0 and 1, and the sum of the weights of all positions is 1, indicating the degree of attention paid to each position of the encoder by the current decoding step.

[0141] Step S30322: The hidden states at each position in the hidden state sequence are weighted and summed according to the attention weights to obtain the context vector of the current time step.

[0142] It should be noted that the device performs a weighted summation of the attention weights and the corresponding encoder hidden states to obtain the context vector for the current time step. :

[0143] Context vector The sequence of hidden states output by the encoder is aggregated. The information most relevant to the current decoding step is used to provide context for the decoder to generate the output of the current time step.

[0144] Step S30323: Concatenate the embedding vector corresponding to the previous time step with the context vector of the current time step as the decoder input of the current time step. The embedding vector output by the previous time step is the initial embedding vector or the real embedding vector determined by the forced teaching strategy based on the repaired address sequence.

[0145] It should be noted that during the training phase, the device can adopt a teacher forcing strategy, which means that the input of the decoder at each time step uses the token embedding from the previous time step in the real target address sequence, rather than the token embedding predicted by the model itself.

[0146] For the t-th time step, the device will embed the true embedding vector from the previous time step. context vector of the current time step The data is spliced ​​together to form the decoder input. :

[0147] Here, `embed` is the word embedding function, which maps discrete tokens to continuous vectors. When t=1, the embedding vector from the previous time step is the initial embedding vector, corresponding to a specific starting token. <sos>.

[0148] In this embodiment, by employing a forced teaching strategy, the device can avoid the accumulation of errors caused by early prediction mistakes, thereby accelerating model convergence.

[0149] Step S30324: Input the decoder input and the hidden state of the previous time step of the preset decoder into the bidirectional long short-term memory network layer to obtain the current hidden state of the encoder.

[0150] It should be noted that the preset decoder includes a BiLSTM layer, which is used to process sequence data and capture contextual information.

[0151] For the t-th time step, the device will input the decoder. Hidden state compared to the previous time step The input is fed into a BiLSTM to calculate the hidden state at the current time step. :

[0152] in, These are the learnable parameters of the BiLSTM network, also known as the decoder parameters. BiLSTM, through information transfer in both forward and backward directions, allows the current hidden state to utilize both past and future contextual information, thereby generating a more coherent address sequence. BiLSTM computation can be based on standard LSTM equations, including input gates, forget gates, output gates, and cell state updates, which will not be elaborated upon in this embodiment.

[0153] Step S30325: Perform linear transformation and normalized exponential function processing on the current hidden state to output the embedding vector corresponding to the current time step.

[0154] It should be understood that the device can hide the state of the current time step. The vector is mapped to the vocabulary space through a linear transformation, then converted into a probability distribution through a normalized exponential function, and finally the embedding vector corresponding to the current time step is output.

[0155] Specifically, the device can first calculate the output logits:

[0156] In the formula, and For learnable output layer parameters, The dimension is V (vocabulary size).

[0157] Subsequently, the device uses the softmax function to... Convert to probability distribution :

[0158] In the formula, the probability distribution Each element in the distribution represents the probability that the corresponding token is predicted at the current time step. The device can select the token with the highest probability as the output for the current time step based on this probability distribution, or use a sampling strategy to generate diverse outputs.

[0159] For example, during the current training phase, the device may employ a forced teaching strategy, with the output embedding vector used for input computation and loss function computation in subsequent time steps.

[0160] Step S3033: Use the embedding vector corresponding to the last time step as the repaired address sequence corresponding to the current batch.

[0161] It should be noted that the device can repeat steps S30321 to S30325 until an end marker is generated. <eos>If the preset maximum sequence length (i.e., the sequence length of the sample output sequence) is reached, the embedding vectors output at each time step can form a complete repaired address sequence in the order of generation. This repaired address sequence is compared with the sample output sequence (i.e., the actual repaired address) and can be used to calculate the cross-entropy loss function, thereby guiding the update of model parameters.

[0162] This embodiment pre-decodes a bidirectional long short-term memory network (LSTM) combined with a soft attention mechanism to progressively generate repaired address sequences from the hidden state sequence output by the encoder. The decoder uses the last hidden state in the encoder's output sequence as its initial state. At each time step, it dynamically focuses on address elements in the encoder related to the current decoding position using the soft attention mechanism. Combined with a forced teaching strategy, it generates the output for the current time step, ultimately obtaining the complete repaired address sequence. Through this decoder structure, the device can progressively generate repaired address sequences based on the encoder's output hidden state sequence. Furthermore, during decoding, the soft attention mechanism enables the model to dynamically focus on key information related to the current generation position in the input address, the bidirectional LSM network allows the model to fully utilize contextual information to generate coherent address outputs, and the forced teaching strategy ensures the stability and convergence efficiency of the training process.

[0163] Based on the first to third embodiments of this application, in the fourth embodiment of this application, the content that is the same as or similar to the first to third embodiments described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the address data repair method of this application.

[0164] In this embodiment, to further illustrate how to apply the previously trained address data repair model for inference, step S40 specifically includes: steps S401~S404: Step S401: Input the current address to be repaired into the trained address data repair model.

[0165] It should be noted that after the model training is complete, the device can repair low-quality addresses that need to be fixed in practical applications. As the current address to be repaired, it is input into the trained address data repair model.

[0166] The addresses currently awaiting repair can originate from application scenarios such as the delivery addresses filled in by users in the logistics and distribution system, the delivery addresses entered by users when registering on e-commerce platforms, or address information recorded in historical archives in the urban planning management system.

[0167] Step S402: Encode the current address to be repaired using the preset encoder that has been trained to obtain the current hidden state sequence.

[0168] It should be noted that the device can display the current address to be repaired. The input is fed into the pre-trained encoder. The encoder encodes the input addresses following a similar process to the training described above: First, the embedding layer encodes the hierarchical positions of each element in the input sequence, and then fuses these hierarchical positions with word embeddings to obtain the input embedding matrix. Next, the encoder layer captures the dependencies between elements at different levels using a multi-head self-attention mechanism, and filters out irrelevant information using a gating mechanism. Finally, the encoder output layer outputs the current hidden state sequence. This hidden state sequence serves as the input to the pre-trained decoder for subsequent address generation.

[0169] Step S403: Decode the current hidden state sequence using the trained preset decoder, and sample the decoding result using a beam search algorithm to obtain several candidate repaired address sequences.

[0170] It should be noted that the device can input the current hidden state sequence output by the preset encoder into the preset decoder after training.

[0171] It should also be noted that, unlike the forced teaching strategy used in the training phase, the pre-defined decoder in the inference phase cannot obtain real sample output sequences as input, so it can gradually generate output sequences based on the model's own predictions.

[0172] It should be understood that, to improve the accuracy of the repair results, the device can use a beam search algorithm to sample the decoding process. Beam search is a heuristic search algorithm that retains multiple candidate sequences with the highest probability at each decoding time step, rather than just retaining a single optimal sequence.

[0173] It should be understood that the device can set the bundle width parameter k to control the number of candidate sequences retained in each step. During the decoding process, after the model generates k candidate sequences through bundle search, each candidate repair sequence... Each of these can correspond to a probability distribution. This probability distribution is calculated based on the softmax function:

[0174] In the formula, This is the current address that needs to be repaired. It is the cumulative logits score of the m-th candidate repair sequence, m [1, k], determined by the logits values ​​output at each time step in the candidate repair sequence. It is obtained by summing the logits values ​​of the corresponding tokens.

[0175] In the specific implementation, at each decoding time step, the device can expand each currently retained candidate repair sequence with all possible next tokens, calculate the probability of the expanded sequence, and select the k sequences with the highest probabilities as candidates for the next step. The decoding process continues until all candidate repair sequences generate end markers or reach a preset maximum sequence length (e.g., based on the maximum length of the sample output sequence). Through a beam search algorithm, the device can obtain k high-probability candidate repaired address sequences, providing a candidate set for subsequent rule filtering. The beam width k is set between 5 and 10 to balance candidate diversity and computational efficiency.

[0176] Step S404: Apply address validity rules to each of the candidate repaired address sequences for filtering and scoring, and select the candidate repaired address sequence with the highest score as the target repaired address.

[0177] It should be noted that the device can apply predefined address validity rules to filter and score multiple candidate repaired address sequences generated by the cluster search.

[0178] Address validity rules can include multiple dimensions such as format compliance verification, geographic existence verification, and semantic consistency verification. Format compliance verification checks whether candidate addresses conform to standard address format specifications, such as whether they contain necessary hierarchical fields and whether the field order is correct. Geographic existence verification verifies whether the geographic entities in the candidate address actually exist, which can be achieved by calling the geographic information system interface or querying the standard address database. Semantic consistency verification checks whether the logical relationships between elements at different levels in the candidate address are correct, such as whether a district-level administrative division belongs to its corresponding city-level administrative division.

[0179] Specifically, the device can apply predefined address validity rule functions. Each candidate repaired address sequence is filtered and scored. This function integrates rules such as format compliance (e.g., zip code length verification), geographic existence (queried via API), and semantic consistency. The score is calculated as follows:

[0180] in, The score is a standardized score in the range of 0-1, with higher scores indicating valid addresses. Therefore, the optimal candidate, i.e., the candidate repair address sequence with the highest score, can be selected.

[0181] In practice, the device calculates a validity score for each candidate address based on the verification results; a higher score indicates better validity of the candidate address. The device then selects the candidate repair address sequence with the highest score. The final corrected address is output. Additionally, if multiple candidate addresses have the same score, the candidate with the highest model confidence is selected first.

[0182] Through the above reasoning process, the device can generate multiple candidate repair results from low-quality input addresses based on the trained address data repair model, and filter them using address validity rules, thereby outputting repaired addresses that conform to actual geographical specifications.

[0183] Furthermore, to further improve the model's repair capability and generalization performance, an iterative optimization mechanism can be introduced. Therefore, after step S404, steps S501~S503 are also included: Step S501: When the target repaired address is verified, the current address to be repaired and the target repaired address are constructed into a correct repair data pair.

[0184] It should be noted that the equipment can confirm the repair results through manual review or automated verification. When the target address is repaired... Upon verification that the repair is correct, the device constructs a correct repair data pair by matching the current address to be repaired with the corresponding target repaired address. This data represents cases where the model successfully repaired the problem and can serve as high-quality samples for subsequent training.

[0185] Step S502: Incorporate the corrected data into the training data pair to obtain the updated training data pair.

[0186] It should be understood that the device can correctly repair the data. Add to the original training data pairs to form the updated training dataset. By continuously accumulating correctly repaired cases, the training dataset is improved. The scale and diversity of data have been continuously expanded, especially in supplementing address error types that are not covered by the original data augmentation in real-world application scenarios.

[0187] Step S503: Use the updated training data to repair the model based on the optimized address data.

[0188] It should be noted that the device uses updated training data for... The trained address data repair model is then further optimized. The optimization process is consistent with the training workflow described above, including dividing the training data pairs into batches, performing forward propagation through the encoder and decoder, calculating cross-entropy loss, and performing backpropagation to update parameters. Through iterative optimization over multiple training cycles, the model can learn more address repair patterns from newly added correct repair cases, continuously improving repair accuracy and generalization ability.

[0189] Specifically, using the updated If the model is retrained, the optimized cross-entropy loss function can be expressed as:

[0190] in, It's the updated version. The number of samples in These are real repair tags (corresponding to the repaired address sample). This is the input address (corresponding to the sample address to be repaired). The model parameters are updated using the Adam optimizer. Then the iterative formula is:

[0191] in, This is the learning rate (initial value 0.001). This represents the number of training steps, and gradient calculation can be based on backpropagation.

[0192] In practical implementation, the device can periodically execute the above iterative optimization process, forming a closed-loop optimization mechanism of "repair-verification-reinjection-retraining". With the continuous accumulation of correctly repaired data pairs and the continuous iteration of the model, the performance of the address data repair model gradually improves, thus better adapting to the complex and ever-changing needs of low-quality address repair.

[0193] Furthermore, you can also refer to this section. Figure 7 This document describes the entire process of the address data recovery method used in this application. Figure 7 This is a schematic diagram illustrating the entire process of the address data repair method in this application.

[0194] Depend on Figure 7 It can be seen that the address data repair method in this application can be divided into a training phase and an inference phase.

[0195] During the training phase, the collected high-quality address dataset can first undergo data augmentation: by simulating address quality problems that may occur in real-world applications, address samples to be repaired and address samples after repair can be generated for model training, in order to construct training data pairs that cover multiple error types.

[0196] Next, a pre-defined encoder can be built based on Transformer. This pre-defined encoder adopts an improved Transformer architecture, whose overall structure includes an embedding layer, an encoder layer, and an encoder output layer. The embedding layer encodes the hierarchical position of each element in the input sequence and merges this hierarchical position encoding with word embeddings to generate the input embedding. The encoder layer captures the dependencies between elements at different levels using a multi-head self-attention mechanism and filters out irrelevant information using a gating mechanism. The encoder output layer normalizes the output of the encoder layer to obtain the final hidden state sequence. The pre-defined decoder is built on BiLSTM and incorporates a soft attention mechanism to progressively generate the repaired address sequence from the hidden state sequence output by the encoder. The decoder uses the last hidden state in the encoder's output sequence as its initial state. At each time step, it dynamically focuses on address elements in the encoder related to the current decoding position using the soft attention mechanism. Combined with a forced teaching strategy, it generates the output for the current time step, ultimately obtaining the complete repaired address sequence.

[0197] After initializing the training parameters of the address data repair model (model parameters in the preset encoder and preset decoder), the preset encoder and preset decoder can be iteratively trained based on the aforementioned training data, including: processing input, generating output, calculating loss, and updating parameters.

[0198] The training of the address data repair model is completed when the loss meets the convergence condition.

[0199] During the inference phase, the trained address data repair model can be directly applied: the current address to be repaired is input into the trained address data repair model. A beam search algorithm can be used to generate several candidate repaired address sequences; then, rule filtering is applied to select the optimal candidate, i.e., the candidate repaired address sequence with the highest score, as the output.

[0200] In addition, to achieve continuous optimization of the model, the current address to be repaired and the corresponding target repaired address can be combined to form a new correct repair data pair and added to the aforementioned training data pair, thereby using the updated training data pair to further optimize the address data repair model that has been trained.

[0201] In this embodiment, through the above-described reasoning and iterative optimization process, the model reasoning stage can not only output high-quality repair results, but also continuously optimize model performance through a continuous learning mechanism, thereby enabling the model to evolve and improve its performance in real-world application scenarios.

[0202] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the address data repair method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0203] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one…" does not exclude the presence of other elements in the process, method, article, or system that includes that element.

[0204] The above embodiment numbers are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. They are only some embodiments of this application and do not limit the scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.< / eos> < / sos>

Claims

1. A method for repairing address data, characterized in that, The method includes: Obtain an address dataset and perform augmentation processing on the address dataset to obtain training data pairs, wherein the training data pairs include address samples to be repaired and corresponding repaired address samples; An address data repair model is initialized. The address data repair model includes a preset encoder based on Transformer and a preset decoder based on bidirectional long short-term memory network. The preset encoder is used to perform encoding operations on the fusion level position of the input address sequence and dependency modeling based on multi-head self-attention mechanism to obtain the hidden state sequence. The preset decoder is used to perform decoding operations on the hidden state sequence to obtain the repaired address sequence. The address data repair model is trained based on the training data to obtain the trained address data repair model; The current address to be repaired is input into the trained address data repair model to obtain the target repaired address.

2. The method as described in claim 1, characterized in that, The step of obtaining the address dataset and performing augmentation processing on the address dataset to obtain training data pairs includes: Obtain the address dataset and parse each address in the address dataset into a sequence of hierarchical fields; A random masking operation is performed on the hierarchical field sequence, and the selected field is replaced with a mask character according to a preset probability; Perform a redundant word removal operation on the masked hierarchical field sequence to identify and remove redundant words in the masked hierarchical field sequence; Perform an error element replacement operation on the hierarchical field sequence after deleting redundant words, replacing the selected field with a preset error version; The hierarchical field sequence after field replacement is used as the address sample to be repaired, and the hierarchical field sequence is used as the repaired address sample to construct training data pairs.

3. The method as described in claim 1, characterized in that, The step of training the address data repair model based on the training data to obtain the trained address data repair model includes: The training data is divided into several training batches to obtain the sample input sequence and sample output sequence corresponding to each required training batch. The sample input sequence corresponding to the current batch is input into the preset encoder to obtain the hidden state sequence corresponding to the current batch. The hidden state sequence corresponding to the current batch is input into the preset decoder to obtain the repaired address sequence corresponding to the current batch; The cross-entropy loss function is calculated based on the repaired address sequence and the sample output sequence corresponding to the current batch, and the parameters of the address data repair model are updated based on the calculation results. Then, the process returns to the step of inputting the sample input sequence corresponding to the current batch into the preset encoder. The training of the address data repair model is completed when the current batch meets the preset convergence condition.

4. The method as described in claim 3, characterized in that, The preset encoder includes: an embedding layer, an encoder layer, and an encoder output layer; The step of inputting the sample input sequence corresponding to the current batch into the preset encoder to obtain the hidden state sequence corresponding to the current batch includes: The embedding layer performs hierarchical position encoding on each element in the sample input sequence, and combines the hierarchical position encoding of each element with the word embedding of each element to obtain the input embedding of each element and construct the input matrix. The encoder layer obtains an attention output matrix based on the multi-head self-attention mechanism and the input matrix, and performs element-level filtering on the attention output matrix based on a gating mechanism. The hidden state sequence is obtained through the encoder output layer based on the filtering results.

5. The method as described in claim 4, characterized in that, The step of performing hierarchical position encoding on each element in the sample input sequence through the embedding layer, and combining the hierarchical position encoding of each element with the word embedding of each element to obtain the input embedding of each element and construct the input matrix includes: Determine the hierarchical index corresponding to each element in the sample input sequence. The hierarchical index is used to characterize the belonging level of each element in the preset standard address hierarchy structure. Calculate the standard sinusoidal position code corresponding to each element, and combine the standard sinusoidal position code with the level-specific embedding corresponding to the level index to obtain the level position code corresponding to each element. The level-specific embedding is the learnable parameter matrix and bias vector corresponding to each level index. The word embedding of each element is determined based on the sample input sequence, and the hierarchical position code corresponding to each element is added to the word embedding of each element to obtain the input embedding of each element; The input embeddings of each element are integrated to obtain the input matrix.

6. The method as described in claim 4, characterized in that, The step of obtaining the attention output matrix through the encoder layer based on the multi-head self-attention mechanism and the input matrix includes: The query matrix, key matrix, and value matrix are calculated based on the shared weight matrix and the input matrix. Based on the multi-head self-attention mechanism, the query matrix, key matrix, and value matrix corresponding to each attention head are determined. The head output features of each attention head are determined based on the query matrix, key matrix, and value matrix corresponding to each attention head. The head output features of each attention head are spliced ​​together, and an attention output matrix is ​​obtained based on the splicing result and a preset output weight matrix.

7. The method as described in claim 3, characterized in that, The step of inputting the hidden state sequence corresponding to the current batch into the preset decoder to obtain the repaired address sequence corresponding to the current batch includes: The hidden state at the last position in the hidden state sequence is used as the initial hidden state of the decoder; At each time step, the attention weights of each decoding position are determined based on the soft attention mechanism and weighted to obtain the context vector. Combining the context vector and the initial embedding vector, a forced teaching strategy is adopted to obtain the current hidden state of the decoder and determine the embedding vector corresponding to each time step. The embedding vector corresponding to the last time step is used as the repaired address sequence corresponding to the current batch.

8. The method as described in claim 7, characterized in that, The preset decoder includes a bidirectional long short-term memory network layer; The step of determining the attention weights for each decoding position at each time step based on a soft attention mechanism and weighting them to obtain a context vector, and then combining the context vector with the initial embedding vector and employing a forced teaching strategy to obtain the current hidden state of the decoder to determine the embedding vector corresponding to each time step, includes: At the current time step, based on the hidden state of the preset decoder in the previous time step and the hidden state sequence, an attention score is calculated, and an attention weight is obtained according to the attention score. The hidden state of the preset decoder in the previous time step is obtained by updating the initial hidden state of the decoder. The hidden states at each position in the hidden state sequence are weighted and summed according to the attention weights to obtain the context vector of the current time step; The embedding vector corresponding to the previous time step is concatenated with the context vector of the current time step and used as the decoder input of the current time step. The embedding vector output by the previous time step is the initial embedding vector or the real embedding vector determined by the forced teaching strategy based on the repaired address sequence. The decoder input and the hidden state of the preset decoder at the previous time step are input into the bidirectional long short-term memory network layer to obtain the current hidden state of the encoder; The current hidden state is processed by linear transformation and normalized exponential function to output the embedding vector corresponding to the current time step.

9. The method as described in claim 1, characterized in that, The step of inputting the current address to be repaired into the trained address data repair model to obtain the target repaired address includes: Input the current address to be repaired into the trained address data repair model; The current address to be repaired is encoded by the preset encoder that has been trained to obtain the current hidden state sequence; The current hidden state sequence is decoded by the preset decoder that has been trained, and the decoding result is sampled using a beam search algorithm to obtain several candidate repaired address sequences. Each candidate repaired address sequence is filtered and scored using address validity rules, and the candidate repaired address sequence with the highest score is selected as the target repaired address.

10. The method as described in claim 9, characterized in that, After the step of applying address validity rules to each of the candidate repaired address sequences for filtering and scoring, and selecting the candidate repaired address sequence with the highest score as the target repaired address, the method further includes: When the target repaired address passes verification, the current address to be repaired and the target repaired address are constructed into a correct repair data pair; The correctly repaired data is incorporated into the training data pair to obtain the updated training data pair; The updated training data is used to repair the model based on the optimized address data.

Images