Address element parsing method, device and electronic device

Abstract

The present invention provides an address element parsing method, device and electronic device, which relates to the technical field of data processing, including: obtaining address data to be parsed; using a preset Lebert model to perform feature encoding on the address data to be parsed to obtain a target address code; and determining a target parsing result of the address data to be parsed based on the target address code. In the method of the present invention, in the preset Lebert model used for feature encoding of the address data to be parsed, its attention calculation adopts non-parametric relative position encoding, which can effectively enhance the model's ability to identify location information. Therefore, the target parsing result determined based on the target address code can more accurately identify the boundaries and entity types of entities in the address element, thereby alleviating the technical problem of low accuracy of parsing results in existing address element parsing methods.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an address element parsing method, device and electronic device. Background Art

[0002] Address is an important text information in daily life. Many scenarios require address registration, such as e-commerce shopping, food delivery, population census, water, electricity and gas account opening, etc. English addresses usually have a strict writing format (from special to general), and there are usually spaces or other special characters between the address elements at each level to separate them, so that the boundaries of each address element can be well distinguished. However, for Chinese addresses, address elements at different levels are connected together, and there is no obvious boundary between address elements at each level. At the same time, since there are no strict restrictions on the writing format of addresses, there are often some problems with actual addresses, such as: incomplete addresses, redundancy in addresses (the same address elements appear repeatedly), and inaccurate address information.

[0003] Based on this, address element parsing is very necessary. Address element parsing is the process of splitting address text into entities with independent semantics and identifying the types of these entities. Existing address element parsing systems mainly use LSTM to encode features and then use CRF to decode them. However, the above methods often cannot well identify the boundaries and entity types of entities in address elements, making it difficult to ensure the accuracy of address element parsing results. Summary of the invention

[0004] The purpose of the present invention is to provide an address element parsing method, device and electronic device to alleviate the technical problem of low accuracy of parsing results in existing address element parsing methods.

[0005] In a first aspect, the present invention provides an address element parsing method, comprising: obtaining address data to be parsed; performing feature encoding on the address data to be parsed using a preset Lebert model to obtain a target address code; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position code; determining a target parsing result of the address data to be parsed based on the target address code; wherein the target parsing result comprises: the starting position of each entity, the ending position of each entity and the type of each entity, wherein the entity represents a segmentation result with independent semantics in the address data to be parsed.

[0006] In an optional implementation, obtaining the address data to be resolved includes: obtaining original address data; filtering invalid characters in the original address data to obtain the address data to be resolved; wherein the invalid characters include: web page links, codes, and punctuation marks.

[0007] In an optional embodiment, the method also includes: obtaining a target training sample; wherein the target training sample includes multiple training address data; performing a masking operation on each of the training address data to obtain a masked training sample; using the masked training sample to train the initial Bert model to obtain a target Bert model; adding an adapter Adapter between the Encoder layers of the target Bert model to obtain a target Lebert model; wherein the Adapter is used to fuse address element information and external vocabulary information; adjusting the encoding method of the attention calculation in the target Lebert model to a non-parametric relative position encoding to obtain the preset Lebert model.

[0008] In an optional embodiment, obtaining a target training sample includes: obtaining a plurality of sample address data; performing data cleaning on each of the sample address data to obtain cleaned sample address data; performing entity type recognition on each of the cleaned sample address data to obtain address data with an entity type mark; performing data enhancement on each of the address data with an entity type mark to obtain the target training sample.

[0009] In an optional embodiment, data enhancement is performed on each of the address data with entity type tags, including: performing entity replacement on the target entity in each of the address data with entity type tags based on a first preset probability distribution; wherein the target entity represents any entity in the address data with entity type tags; the first preset probability distribution includes: a first probability and a second probability; the first probability is the probability that the entity remains unchanged, and the second probability is the probability that the entity is replaced; the sum of the first probability and the second probability is 1; the second probability specifically includes a second preset probability distribution, and the second preset probability distribution includes: a first sub-probability, a second sub-probability and a third sub-probability; the first sub-probability is the probability that the entity is randomly replaced, the second sub-probability is the probability that the entity is deleted, and the third sub-probability is the probability that the entity remains unchanged; the sum of the first sub-probability, the second sub-probability and the third sub-probability is 1.

[0010] In an optional embodiment, a masking operation is performed on each of the training address data, including: masking a preset proportion of characters in each of the training address data based on a third preset probability distribution to obtain a masked training sample; wherein the third preset probability distribution includes: a third probability, a fourth probability and a fifth probability; the third probability is the probability that the character is replaced by a preset masked character, the fourth probability is the probability that the character remains unchanged, and the fifth probability is the probability that the character is randomly replaced; the sum of the third probability, the fourth probability and the fifth probability is 1.

[0011] In an optional implementation, the attention calculation in the preset Lebert model adopts the following formula: Among them, q i The query vector representing the i-th element in the address data, x i represents the feature obtained after the i-th element is mapped by the embedding layer in the preset Lebert model, W Q represents the first preset parameter matrix, W K represents the second preset parameter matrix, W V represents the third preset parameter matrix, x j represents the feature obtained after the jth element in the address data is mapped by the embedding layer in the preset Lebert model, k j represents the key vector of the j-th element, v j Represents the value vector of the j-th element, a i,j represents the attention score of the i-th element and the j-th element, o i Represents the feature vector of the i-th element after attention, R i,j represents the relative position code of the i-th element and the j-th element, and R i,j Calculate using the following formula: Where k represents the index of the kth position in the hidden vector, and d z represents the dimension of the hidden vector.

[0012] In an optional embodiment, determining a target parsing result of the address data to be parsed based on the target address code includes: processing the target address code using a preset pointer network to obtain an initial parsing result of the address data to be parsed; wherein the initial parsing result includes all optional entities in the address data to be parsed; determining whether there is entity nesting among all the optional entities; wherein the entity nesting indicates that there are overlapping characters between entities; if so, processing the initial parsing result based on preset rules to obtain the target parsing result; wherein the preset rules include: if two adjacent entities are of the same entity type, retaining the entity with the first character appearing relatively earlier; if two adjacent entities are of different entity types, correcting the end position of the first entity to be the one before the start position of the second entity; wherein the first entity is an entity with the first character appearing relatively earlier in the two adjacent entities; and the second entity is an entity with the first character appearing later in order than the first entity.

[0013] In a second aspect, the present invention provides an address element parsing device, comprising: a first acquisition module, used to acquire address data to be parsed; an encoding module, used to perform feature encoding on the address data to be parsed using a preset Lebert model to obtain a target address encoding; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position encoding; a determination module, used to determine a target parsing result of the address data to be parsed based on the target address encoding; wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents a segmentation result with independent semantics in the address data to be parsed.

[0014] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the steps of the method described in any one of the aforementioned embodiments are implemented.

[0015] The address element parsing method provided by the present invention comprises: obtaining address data to be parsed; performing feature encoding on the address data to be parsed using a preset Lebert model to obtain a target address code; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position code; and determining a target parsing result of the address data to be parsed based on the target address code; wherein the target parsing result comprises: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents a word segmentation result with independent semantics in the address data to be parsed.

[0016] In the method of the present invention, in the preset Lebert model used for feature encoding of the address data to be parsed, its attention calculation adopts non-parametric relative position encoding, which can effectively enhance the model's ability to recognize location information. Therefore, the target parsing result determined based on the target address encoding can more accurately identify the boundaries and entity types of entities in the address elements, thereby alleviating the technical problem of low accuracy of parsing results in existing address element parsing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of an address element parsing method provided by an embodiment of the present invention;

[0019] Figure 2 A flowchart for determining a preset Lebert model provided by an embodiment of the present invention;

[0020] Figure 3 An architectural diagram of an address element parsing system composed of a preset Lebert model and a preset pointer network provided in an embodiment of the present invention;

[0021] Figure 4 A functional module diagram of an address element parsing device provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0026] Common addresses generally contain the following types of information: 1) administrative division information, such as province, city, county, and township information; 2) road network information, such as road names, road numbers, and road facilities; 3) detailed address information, such as POI (Point Of Interest), building number, and room number; 4) non-address information, such as supplementary instructions, incorrect input, etc. Address element parsing is the process of splitting the address text into entities with independent semantics and identifying the types of these entities. That is, perform word segmentation operations on the address to obtain multiple word segmentation results of the address, and each word segmentation result is regarded as an entity.

[0027] For example, input: "No. 969, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province, Building 5, Taobao City, placed at the front desk", the address element parsing system should output "Province = Zhejiang Province city = Hangzhou City district = Yuhang District town = Wuchang Street road = Wenyi West Road road_number = No. 969 poi = Taobao City house_number = Building 5 O =, placed at the front desk".

[0028] In Chinese addresses, there are complex dependencies between the categories to which each address element belongs. The same address element may appear in different locations in an address and may belong to different categories. However, in real life, the first half of the address usually follows a strict writing format, such as the more standardized writing of provinces, cities and districts. However, the second half of the address usually has a more flexible writing format, such as writing the name of the district directly after the district, or describing it in a natural language. The existing address element parsing system mainly uses LSTM to encode features and then uses CRF to decode them. However, the above method often cannot identify the boundaries and entity types of entities in the address elements well, and it is difficult to ensure the accuracy of the address element parsing results. In view of this, an embodiment of the present invention provides an address element parsing method to alleviate the technical problems mentioned above.

[0029] Embodiment 1

[0030] Figure 1A flowchart of an address element parsing method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method specifically comprises the following steps:

[0031] Step S102, obtaining address data to be resolved.

[0032] Step S104, using a preset Lebert model to perform feature encoding on the address data to be parsed, to obtain a target address code.

[0033] The address data to be parsed in the embodiment of the present invention is the address data after data cleaning. Data cleaning is to remove abnormal data in the original address data. After the address data to be parsed is obtained, the address data to be parsed is input into a preset Lebert model to perform feature encoding on the address data to be parsed to obtain a target address encoding, which can also be regarded as a feature vector of the address data to be parsed. The Lebert model is a data model for encoding words to obtain word features.

[0034] The existing Lerbert model uses parametric absolute position coding for attention calculation. The applicant has found that when the attention scores of two elements in the address data are calculated using absolute position coding, the data that can reflect the relative relationship between the two elements will be lost. Therefore, in an embodiment of the present invention, the attention calculation in the preset Lebert model uses non-parametric relative position coding to enhance the ability of the Lebert model to recognize location information.

[0035] Step S106, determining a target resolution result of the address data to be resolved based on the target address code.

[0036] After obtaining the target address code of the address data to be parsed, the target parsing result of the address data to be parsed can be further obtained through decoding operation, wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents the word segmentation result with independent semantics in the address data to be parsed.

[0037] The address element parsing method provided by the embodiment of the present invention includes: obtaining address data to be parsed; using a preset Lebert model to perform feature encoding on the address data to be parsed to obtain a target address code; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position code; determining a target parsing result of the address data to be parsed based on the target address code; wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents a segmentation result with independent semantics in the address data to be parsed.

[0038] In the method of the present invention, in the preset Lebert model used for feature encoding of the address data to be parsed, its attention calculation adopts non-parametric relative position encoding, which can effectively enhance the model's ability to recognize location information. Therefore, the target parsing result determined based on the target address encoding can more accurately identify the boundaries and entity types of entities in the address elements, thereby alleviating the technical problem of low accuracy of parsing results in existing address element parsing methods.

[0039] In an optional implementation, the above step S102, obtaining the address data to be resolved, specifically includes the following steps:

[0040] Step S1021, obtaining original address data;

[0041] Step S1022, filtering invalid characters in the original address data to obtain address data to be parsed.

[0042] Specifically, due to the diversity of data sources of the original address data, there may be dirty data (abnormal data) in the original address data, for example, address links mixed with pictures, some invalid HTML codes, or some invalid punctuation marks. These abnormal data will affect the result of model feature encoding. Therefore, before using the above preset Lebert model to feature encode the address data, it is necessary to filter out the invalid characters contained therein, where invalid characters include: web links, codes and punctuation marks. Filtering can choose to use regular expressions to process the address data.

[0043] It is introduced above that the implementation of the present invention is to use the preset Lebert model to perform feature encoding on the address data to be parsed, and the key differences between the preset Lebert model and the existing model. The following will introduce in detail the process of how to obtain the preset Lebert model.

[0044] In an optional embodiment, if Figure 2 As shown, the method of the present invention also includes the following steps:

[0045] Step S201, obtaining target training samples.

[0046] In order to obtain a preset Lebert model, an embodiment of the present invention first pre-trains the Bert model, and then fine-tunes the model to obtain the preset Lebert model. When pre-training the Bert model, first obtain a target training sample, wherein the target training sample includes a plurality of training address data; each training address data is address data with an entity type tag and a starting position and an ending position of each entity. That is, the address text and the address text label together constitute the training address data, for example, "Liansheng Road, Xihu District, Hangzhou City, Zhejiang Province" and "B-prov I-prov E-prov B-city I-city E-city B-district I-district E-district B-road I-road E-road" are a training address data, wherein B represents the beginning of an address entity, I represents the internal position of the address entity, and E represents the end of the address entity.

[0047] Step S202, performing a masking operation on each training address data to obtain a masked training sample.

[0048] After obtaining multiple training address data, in order to judge the accuracy of the Bert model output data, before training the Bert model, it is necessary to perform a mask operation on each training address data, that is, mask some characters in the training address data, and then input the masked training samples into the model for training.

[0049] Step S203: train the initial Bert model using the masked training samples to obtain a target Bert model.

[0050] At present, most of the address element parsing models are fine-tuned based on the model weights obtained by pre-training on general corpus. However, there are still relatively large differences between general corpus and address data. Although fine-tuning directly based on the pre-trained model obtained by pre-training on general corpus can achieve relatively good results, the model performance has not been fully explored. Therefore, the embodiment of the present invention uses the pre-trained model obtained by training on general corpus to continue pre-training on address data to improve the performance of the address element parsing model. In the embodiment of the present invention, the weights of the initial Bert model are initialized using the weights of the model obtained by training on general corpus, and then the initial Bert model is trained using the masked training samples to obtain the target Bert model.

[0051] Step S204, adding an adapter between the Encoder layers of the target Bert model to obtain a target Lebert model.

[0052] If the target Bert model is directly used as the encoding module of the address element parsing system, the following problems will exist during address element parsing: 1) The recognition of the address entity boundary is inaccurate; 2) The recognition of the address entity type is inaccurate. In order to enhance the model's ability to recognize entity boundaries and entity types, after obtaining the target Bert model, an Adapter is introduced between the Encoder layers of the target Bert model to fuse address element information and external vocabulary information, so as to obtain the target Lebert model. That is to say, the role of the Adapter is to fuse address element information and external vocabulary information, and the target Lebert model is an improved model based on the Bert model.

[0053] Before the target Lebert model processes the address data, first construct a trie tree using external vocabulary information, and then obtain the top k external vocabulary information related to each element in the address by querying the trie tree. For a given Chinese address sc = {c_1, c_2, c_3, c_4}, use the trie tree to match the external vocabulary information for each element in sc, and combine sc and the matched external vocabulary information in the form of character-vocabulary pairs, denoted as sc_w = {(c_1, ws_1), (c_2, ws_2), (c_3, ws_3), (c_4, ws_4)}, where c_i represents the i-th character in the Chinese address sc, and ws_i represents the set of external vocabulary words matched by the i-th character.

[0054] For example, the Chinese address is "Hangzhou City, Zhejiang Province", and it is assumed that up to 5 external vocabulary words are matched for each character through the trie tree. Taking c_1 = "浙" as an example, the possible matched external vocabulary words ws_1 = ["浙江", "浙江省", "浙江省杭州市", "浙江省丽水市", "浙江省宁波市"].

[0055] In the Lebert model, use to represent the feature of the i-th character in the address, where c represents the number of layers in the encoder layer of the Lebert model; represents the set / sequence of word vectors of the external vocabulary words corresponding to the i-th character (the word vector of each external vocabulary word can be obtained using the word2vec model), ws represents the sequence, where w is the abbreviation of word, represents the word vector of the m-th word in the external vocabulary sequence matched by the i-th character, and g represents the total number of external vocabulary words in the word vector set.

[0056] Since the feature of the i-th character is trained by the Lebert model, while the word vectors of external vocabulary are trained by other models (such as the word2vec model). Therefore, the model difference will lead to feature dimension (e.g. 1024) and The feature dimensions (such as 128 or 256, etc.) are not aligned. Therefore, in order to perform attention operations to calculate similarity, it is necessary to align the dimensions through nonlinear transformation. The dimension alignment is calculated using the following formula in, Yes The vector obtained by dimension alignment is W1, W2 represent the parameter matrix of the fully connected layer in the neural network, b1, b2 represent the bias, d c for The characteristic dimension, d w for The feature dimension of , using the above formula to calculate the vector of all external vocabulary word vectors in the external vocabulary set after dimension alignment, and then get the dimension-aligned word vector set

[0057] Next, use the attention mechanism to and Y i Fusion is performed and similarity scores are calculated using bilinear transformation Among them, the similarity score a i is a sequence, W attn Represents the preset attention parameter matrix, and then uses the similarity score a i Y i Perform weighted summation to obtain the fused word features Among them, a i,j Indicates a i The score of the jth position in , Represents Y i The word vector at the jth position in the word, and finally the character feature And the fused word features Add together to get After getting the word features after addition, use replace Input to the next layer encoder of the Lebert model.

[0058] Step S205, adjusting the encoding method of attention calculation in the target Lebert model to non-parametric relative position encoding to obtain a preset Lebert model.

[0059] As described above, there are substantial differences in the encoding methods between the preset Lebert model in the embodiment of the present invention and the attention calculation of the existing Lebert model. Therefore, after obtaining the target Lebert model, its encoding method is adjusted to obtain the preset Lebert model.

[0060] In an optional implementation, the attention calculation in the preset Lebert model uses the following formula: Among them, q i The query vector representing the i-th element in the address data, x i W represents the feature obtained after the i-th element is mapped by the embedding layer in the preset Lebert model. Q represents the first preset parameter matrix, W K represents the second preset parameter matrix, W V represents the third preset parameter matrix, x j represents the feature obtained after the jth element in the address data is mapped by the embedding layer in the preset Lebert model, k j represents the key vector of the jth element, v j Represents the value vector of the jth element, a i,j represents the attention score of the i-th element and the j-th element, o i Represents the feature vector of the i-th element after attention, R i,j represents the relative position encoding of the i-th element and the j-th element, and R i,j Calculate using the following formula: Where k represents the index of the kth position in the hidden vector, and d z Represents the dimension of the hidden vector, the value is 1024.

[0061] In an optional implementation, the above step S201, obtaining a target training sample, specifically includes the following steps:

[0062] Step S2011, obtaining a plurality of sample address data.

[0063] Step S2012, performing data cleaning on each sample address data to obtain cleaned sample address data.

[0064] Specifically, when preparing sample data, sample address data can be crawled from the Internet by crawling, but sometimes there are many abnormal data in the sample address data crawled by the crawler, such as the address link of the picture or some other invalid HTML code mixed in the address data, and sometimes there are some invalid punctuation marks in the address element. For these abnormal data, the invalid characters can be filtered out by using regular expressions during the data cleaning process, and then the cleaned sample address data can be obtained.

[0065] Step S2013, performing entity type recognition on each cleaned sample address data to obtain address data with entity type tags.

[0066] After obtaining the cleaned sample address data, the address elements can be labeled through manual crowdsourcing, that is, the entity boundaries and entity types of the address data can be identified to obtain address data with entity type labels.

[0067] In the embodiment of the present invention, the address elements can be divided into the following label types: 1.prov: provincial administrative divisions, provinces, autonomous regions, municipalities directly under the central government; 2.city: prefecture-level administrative divisions, prefecture-level cities, regions, autonomous prefectures, etc.; 3.district: county-level administrative divisions, urban districts, county-level cities, counties; 4.devzone: development zones in a broad sense; 5.town: township-level administrative divisions, towns, streets, townships, etc.; 6.community: including communities, administrative villages (production brigades, village committees), and natural villages; 7.village_group: limited to xx group, xx team, xx community (xx is a number); 8.ro ad: roads with formal names, including tunnels, elevated roads, streets, lanes, alleys, pedestrian streets, commercial streets, etc.; 9.roadno: road number; 10.poi: point of interest, generally a single-family building on the map; 11.subpoi: sub-point of interest; 12.houseno: building number; 13.cellno: cell number; 14.floorno: floor number; 15.roomno: room number; 16.assist: auxiliary positioning word; 17.distance: distance auxiliary positioning word; 18.intersection: road intersection, intersection, road entrance and exit; 19: O: non-address element.

[0068] After determining the label type, you can choose to use BIOS marking or BIO marking to mark each entity. If you use BIOS marking, if the address is "Hangzhou City, Zhejiang Province", then the corresponding mark is "B-provI-prov E-prov B-city I-city E-city", where B represents the beginning of an address entity, I represents the internal position of the address entity, and E represents the end of the address entity.

[0069] Step S2014, performing data enhancement on each address data with entity type mark to obtain target training samples.

[0070] Since training deep learning models requires a large amount of labeled data, but the crawled sample address data is limited, and the cost of labeling through manual crowdsourcing is expensive, some data enhancement methods are needed to increase the training data, so as to obtain target training samples with a larger amount of data.

[0071] In an optional implementation, the above step S2014 performs data enhancement on each address data with an entity type tag, specifically including the following contents:

[0072] Based on a first preset probability distribution, entity replacement is performed on a target entity in each address data having an entity type tag; wherein the target entity represents any entity in the address data having an entity type tag.

[0073] The first preset probability distribution includes: a first probability and a second probability; the first probability is the probability that the entity remains unchanged, and the second probability is the probability that the entity is replaced; the sum of the first probability and the second probability is 1.

[0074] The second probability specifically includes a second preset probability distribution, and the second preset probability distribution includes: a first sub-probability, a second sub-probability and a third sub-probability; the first sub-probability is the probability that the entity is randomly replaced, the second sub-probability is the probability that the entity is deleted, and the third sub-probability is the probability that the entity remains unchanged; the sum of the first sub-probability, the second sub-probability and the third sub-probability is 1.

[0075] For natural language processing, in the scenario of address element parsing, in order not to destroy entity information and sentence semantic information, data enhancement can be performed by randomly replacing entities of the same type. However, this method of randomly replacing entities of the same type for data enhancement may cause the model to overfit.

[0076] In order to solve this problem, the embodiment of the present invention no longer performs random replacement when replacing entities of the same type, but introduces a first preset probability distribution to replace the entity. For ease of understanding, the following example illustrates that if the first probability is 50%, the second probability is 50%, the first sub-probability is 60%, the second sub-probability is 30%, and the third sub-probability is 10%. This means that an entity in the address may be replaced with a probability of 50%, or it may not be replaced with a probability of 50%; if the entity needs to be replaced, then the same type of entity is randomly replaced with a probability of 60%, deleted with a probability of 30%, and remains unchanged with a probability of 10%.

[0077] For example, the address "Good Time Youth Hotel Apartment, Liansheng Road, Yuhang District, Hangzhou City, Zhejiang Province", the entity label corresponding to the address is "B-prov I-prov E-prov B-city I-city E-city B-district I-district E-district B-road I-road E-road B-poi I-poi I-poiI-poi I-poi I-poi I-poi I-poi I-poi E-poi", assuming that "Zhejiang Province" and "Hangzhou City" are not replaced, "Yuhang District" is replaced with the same type of entity "Xihu District", and "Good Time Youth Hotel Apartment" is deleted. Then after data enhancement, the address "Liansheng Road, Xihu District, Hangzhou City, Zhejiang Province" will be obtained, and the corresponding label is "B-prov I-prov E-prov B-city I-city E-city B-district I-district E-district B-road I-road E-road".

[0078] In an optional implementation, the above step S202, performing a masking operation on each training address data, specifically includes the following contents:

[0079] Based on the third preset probability distribution, characters in a preset proportion in each training address data are masked to obtain masked training samples.

[0080] Among them, the third preset probability distribution includes: a third probability, a fourth probability and a fifth probability; the third probability is the probability that the character is replaced by a preset masked character, the fourth probability is the probability that the character remains unchanged, and the fifth probability is the probability that the character is randomly replaced; the sum of the third probability, the fourth probability and the fifth probability is 1.

[0081] For ease of understanding, the masking operation is illustrated below. If the preset ratio is 15%, the third probability is 80%, the fourth probability is 10%, the fifth probability is 10%, and the preset masking character is [MASK], then 15% of the characters in an address will be masked, and then there is an 80% probability that the 15% masked characters will be replaced with [MASK], a 10% probability that they remain unchanged, and a 10% probability that they will be randomly replaced.

[0082] For example, the address "Good Time Youth Hotel, Liansheng Road, Yuhang District, Hangzhou City, Zhejiang Province" has a total of 20 characters, so a total of 20*0.15=3 characters are masked. Assume that the masked characters are "Hang, District, Wine", where "Hang" is replaced with the special character "[MASK]" with a probability of 80%, "District" is randomly replaced with "Shang" with a probability of 10%, and "Wine" remains unchanged with a probability of 10%. Then the pre-training data obtained after masking the address data is "Good Time Youth Hotel, Shangliansheng Road, Yuhang City, Zhejiang Province [MASK]".

[0083] In an optional implementation, the above step S106, determining the target resolution result of the address data to be resolved based on the target address code, specifically includes the following steps:

[0084] Step S1061, using a preset pointer network to process the target address code to obtain an initial parsing result of the address data to be parsed.

[0085] Specifically, since some addresses are heavily colloquial, the CRF decoding method cannot perform address decoding well. Therefore, the embodiment of the present invention uses a preset pointer network combined with preset rules to decode the target address code. Among them, the preset pointer network is composed of two fully connected layers, the first fully connected layer is called the start pointer, and the second fully connected layer is called the end pointer. The function of the pointer network is to predict the starting position and end position of an address entity in the address and the type of the entity. Since there may be nested entities in the address elements, the preset pointer network will output all possible entities to which the address elements belong, and then form the initial parsing result of the address data to be parsed, that is, the initial parsing result includes all optional entities in the address data to be parsed.

[0086] Step S1062, determining whether there is entity nesting among all optional entities.

[0087] Since each address element can only belong to one entity type in address feature parsing, such as Hangzhou City, Zhejiang Province, the corresponding unique labels are province and city. Therefore, after obtaining the initial parsing results, it is necessary to determine whether there is entity nesting in all optional entities. If there is entity nesting in the initial parsing results, it is necessary to use preset rules for post-processing, where entity nesting means that there are overlapping characters between entities; for example, the address "Putian City Licheng District Huangshi Town Shaban Village Expressway Putian Taxation Administration Office Compound", where "expressway" is an entity, "Putian Taxation Administration Office Compound" is an entity, "office compound" is also an entity, "Putian Taxation Administration Office Compound" and "office compound" are cases of entity nesting; the starting position of the first entity is 3 and the ending position is 7, the starting position of the latter entity is 6 and the ending position is 9, so these two entities are also nested.

[0088] If it exists, execute the following step S1063; if it does not exist, use the initial parsing result as the target parsing result.

[0089] Step S1063, processing the initial analysis result based on preset rules to obtain a target analysis result.

[0090] The preset rules include:

[0091] If two adjacent entities are of the same entity type, the entity with the first character appearing earlier is retained.

[0092] If two adjacent entities are of different entity types, the end position of the first entity is corrected to be the position before the start position of the second entity; wherein the first entity is the entity whose first character appears relatively earlier in the order of the two adjacent entities; and the second entity is the entity whose first character appears later in the order of the first entity.

[0093] Specifically, there are two situations in entity nesting. Situation 1: The model recognition type of the previous entity and the next entity is the same. Situation 2: The model recognition type of the previous entity and the next entity is different. When encountering situation 1, the previous entity is retained. For example, "Putian Taxation Administration Office Compound" and "Office Compound" in the previous text, "Putian Taxation Administration Office Compound" will be retained in the target parsing result; when encountering situation 2, it is necessary to correct the end index position of the previous entity. For example, the start position of the previous entity is 1 and the end position is 8, and the start position of the next entity is 5 and the end position is 8, then correct the end position of the first entity to 4.

[0094] Figure 3 The architecture diagram of an address element parsing system composed of a preset Lebert model and a preset pointer network provided by an embodiment of the present invention is shown in FIG. Figure 3 , suppose the label of province is set to 1, the label of city is set to 2, and the label of district is set to 3. Because the start pointer outputs the label 1 at the second position and the end pointer outputs the label 1 at the fourth position, we know that positions 2-4 in the original input address represent a province entity, and the same applies to the others.

[0095] In summary, the address element parsing method provided by the embodiment of the present invention, when encoding address data, the preset Lebert model enhances the model's ability to identify entity boundaries and entity types by introducing external vocabulary information, and enhances the model's ability to identify location information by relative position encoding; in addition, the decoding method using a preset pointer network and preset rule processing is used to solve the problem of address entity nesting. Therefore, the method of the present invention can more accurately identify the boundaries and entity types of entities in the address element, effectively alleviating the technical problem of low accuracy of parsing results in the existing address element parsing method.

[0096] Embodiment 2

[0097] The embodiment of the present invention further provides an address element parsing device, which is mainly used to execute the address element parsing method provided in the above-mentioned embodiment 1. The address element parsing device provided in the embodiment of the present invention is specifically introduced below.

[0098] Figure 4 is a functional module diagram of an address element parsing device provided by an embodiment of the present invention, such as Figure 4 As shown, the device mainly includes: a first acquisition module 10, an encoding module 20, a determination module 30,

[0099] in:

[0100] The first acquisition module 10 is used to acquire address data to be resolved.

[0101] The encoding module 20 is used to perform feature encoding on the address data to be parsed using a preset Lebert model to obtain a target address encoding; wherein the attention calculation in the preset Lebert model adopts non-parametric relative position encoding.

[0102] The determination module 30 is used to determine the target parsing result of the address data to be parsed based on the target address code; wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents the word segmentation result with independent semantics in the address data to be parsed.

[0103] The address element parsing device provided by the embodiment of the present invention includes: a first acquisition module 10, used to obtain address data to be parsed; an encoding module 20, used to use a preset Lebert model to perform feature encoding on the address data to be parsed to obtain a target address encoding; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position encoding; a determination module 30, used to determine a target parsing result of the address data to be parsed based on the target address encoding; wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents a segmentation result with independent semantics in the address data to be parsed.

[0104] In the device provided by the embodiment of the present invention, in the preset Lebert model used for feature encoding of the address data to be parsed, its attention calculation adopts non-parametric relative position encoding, and the relative position encoding can effectively enhance the ability of the model to recognize location information. Therefore, the target parsing result determined based on the target address encoding can more accurately identify the boundaries and entity types of entities in the address elements, thereby alleviating the technical problem of low accuracy of parsing results in existing address element parsing methods.

[0105] Optionally, the first acquisition module 10 is specifically used for:

[0106] Obtaining original address data; filtering invalid characters in the original address data to obtain address data to be parsed; wherein invalid characters include: web page links, codes and punctuation marks.

[0107] Optionally, the device further comprises:

[0108] The second acquisition module is used to acquire a target training sample; wherein the target training sample includes a plurality of training address data.

[0109] The masking module is used to perform a masking operation on each training address data to obtain a masked training sample.

[0110] The training module is used to train the initial Bert model using the masked training samples to obtain the target Bert model.

[0111] An add module is used to add an adapter between the Encoder layers of the target Bert model to obtain a target Lebert model; wherein the Adapter is used to fuse address element information and external vocabulary information.

[0112] The adjustment module is used to adjust the encoding method of attention calculation in the target Lebert model to non-parametric relative position encoding to obtain a preset Lebert model.

[0113] Optionally, the second acquisition module includes:

[0114] The acquisition unit is used to acquire a plurality of sample address data.

[0115] The cleaning unit is used to clean each sample address data to obtain cleaned sample address data.

[0116] The identification unit is used to perform entity type identification on each cleaned sample address data to obtain address data with entity type mark.

[0117] The data enhancement unit is used to perform data enhancement on each address data with entity type tags to obtain target training samples.

[0118] Optionally, the data enhancement unit is specifically used for:

[0119] Based on a first preset probability distribution, entity replacement is performed on a target entity in each address data having an entity type tag; wherein the target entity represents any entity in the address data having an entity type tag.

[0120] The first preset probability distribution includes: a first probability and a second probability; the first probability is the probability that the entity remains unchanged, and the second probability is the probability that the entity is replaced; the sum of the first probability and the second probability is 1.

[0121] The second probability specifically includes a second preset probability distribution, and the second preset probability distribution includes: a first sub-probability, a second sub-probability and a third sub-probability; the first sub-probability is the probability that the entity is randomly replaced, the second sub-probability is the probability that the entity is deleted, and the third sub-probability is the probability that the entity remains unchanged; the sum of the first sub-probability, the second sub-probability and the third sub-probability is 1.

[0122] Optionally, the shielding module is specifically used for:

[0123] Based on the third preset probability distribution, characters in a preset proportion in each training address data are masked to obtain masked training samples.

[0124] Among them, the third preset probability distribution includes: a third probability, a fourth probability and a fifth probability; the third probability is the probability that the character is replaced by a preset masked character, the fourth probability is the probability that the character remains unchanged, and the fifth probability is the probability that the character is randomly replaced; the sum of the third probability, the fourth probability and the fifth probability is 1.

[0125] Optionally, the attention calculation in the preset Lebert model uses the following formula: Among them, q i The query vector representing the i-th element in the address data, x iW represents the feature obtained after the i-th element is mapped by the embedding layer in the preset Lebert model. Q represents the first preset parameter matrix, W K represents the second preset parameter matrix, W V represents the third preset parameter matrix, x j represents the feature obtained after the jth element in the address data is mapped by the embedding layer in the preset Lebert model, k j represents the key vector of the jth element, v j Represents the value vector of the jth element, a i,j represents the attention score of the i-th element and the j-th element, o i Represents the feature vector of the i-th element after attention, R i,j represents the relative position encoding of the i-th element and the j-th element, and R i,j Calculate using the following formula: Where k represents the index of the kth position in the hidden vector, and d z Represents the dimension of the hidden vector.

[0126] Optionally, the determination module 30 is specifically configured to:

[0127] The target address encoding is processed using a preset pointer network to obtain an initial parsing result of the address data to be parsed; wherein the initial parsing result includes all optional entities in the address data to be parsed.

[0128] Determine whether there is entity nesting among all optional entities; entity nesting means that there are overlapping characters between entities.

[0129] If it exists, the initial parsing result is processed based on the preset rules to obtain the target parsing result.

[0130] The preset rules include:

[0131] If two adjacent entities are of the same entity type, the entity with the first character appearing earlier is retained.

[0132] If two adjacent entities are of different entity types, the end position of the first entity is corrected to be the position before the start position of the second entity; wherein the first entity is the entity whose first character appears relatively earlier in the order of the two adjacent entities; and the second entity is the entity whose first character appears later in the order of the first entity.

[0133] Embodiment 3

[0134] See also Figure 5An embodiment of the present invention provides an electronic device, which includes: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected via the bus 62; the processor 60 is used to execute an executable module stored in the memory 61, such as a computer program.

[0135] The memory 61 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 63 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0136] The bus 62 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0137] Among them, the memory 61 is used to store programs, and the processor 60 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 60 or implemented by the processor 60.

[0138] The processor 60 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 60. The above processor 60 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 61, and the processor 60 reads the information in the memory 61 and completes the steps of the above method in combination with its hardware.

[0139] An address element parsing method, device, and computer program product of an electronic device provided in an embodiment of the present invention include a computer-readable storage medium storing a non-volatile program code executable by a processor. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0142] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0143] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0144] In addition, the terms "horizontal", "vertical", "overhanging" and the like do not mean that the components are required to be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0145] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for parsing address elements, characterized in that: include: Get the address data to be parsed; The address data to be parsed is feature encoded using a preset Lebert model to obtain a target address code; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position encoding; the attention calculation in the preset Lebert model adopts the following formula: ,in, The query vector representing the i-th element in the address data, represents the feature obtained after the i-th element is mapped by the embedding layer in the preset Lebert model, represents the first preset parameter matrix, represents the second preset parameter matrix, represents the third preset parameter matrix, It represents the feature obtained after the jth element in the address data is mapped by the embedding layer in the preset Lebert model. represents the key vector of the j-th element, represents the value vector of the j-th element, represents the attention score of the i-th element and the j-th element, Represents the feature vector of the i-th element after attention, represents the relative position code of the i-th element and the j-th element, and Calculate using the following formula: , where k represents the index of the kth position in the hidden vector, represents the dimension of the hidden vector; The target parsing result of the address data to be parsed is determined based on the target address code; wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents a word segmentation result with independent semantics in the address data to be parsed.

2. The method according to claim 1, characterized in that Get the address data to be parsed, including: Get the original address data; The invalid characters in the original address data are filtered to obtain the address data to be parsed; wherein the invalid characters include: web page links, codes and punctuation marks.

3. The method according to claim 1, characterized in that The method further comprises: Acquire a target training sample; wherein the target training sample includes a plurality of training address data; Performing a masking operation on each of the training address data to obtain a masked training sample; The initial Bert model is trained using the masked training samples to obtain the target Bert model; Adding an adapter between the Encoder layers of the target Bert model to obtain a target Lebert model; wherein the adapter is used to fuse address element information and external vocabulary information; The encoding method of the attention calculation in the target Lebert model is adjusted to non-parametric relative position encoding to obtain the preset Lebert model.

4. The method according to claim 3, characterized in that Obtain target training samples, including: Get multiple sample address data; Performing data cleaning on each of the sample address data to obtain cleaned sample address data; Performing entity type recognition on each of the cleaned sample address data to obtain address data with entity type tags; Data enhancement is performed on each of the address data with entity type tags to obtain the target training sample.

5. The method according to claim 4, characterized in that Performing data enhancement on each of the address data having the entity type tag, including: Performing entity replacement on a target entity in each of the address data with entity type tags based on a first preset probability distribution; wherein the target entity represents any entity in the address data with entity type tags; The first preset probability distribution includes: a first probability and a second probability; the first probability is the probability that the entity remains unchanged, and the second probability is the probability that the entity is replaced; the sum of the first probability and the second probability is 1; The second probability specifically includes a second preset probability distribution, and the second preset probability distribution includes: a first sub-probability, a second sub-probability and a third sub-probability; the first sub-probability is the probability that the entity is randomly replaced, the second sub-probability is the probability that the entity is deleted, and the third sub-probability is the probability that the entity remains unchanged; the sum of the first sub-probability, the second sub-probability and the third sub-probability is 1.

6. The method according to claim 3, characterized in that: Performing a masking operation on each of the training address data includes: Based on a third preset probability distribution, masking a preset proportion of characters in each of the training address data to obtain masked training samples; Among them, the third preset probability distribution includes: a third probability, a fourth probability and a fifth probability; the third probability is the probability that the character is replaced by a preset masked character, the fourth probability is the probability that the character remains unchanged, and the fifth probability is the probability that the character is randomly replaced; the sum of the third probability, the fourth probability and the fifth probability is 1.

7. The method according to claim 1, characterized in that Determining a target resolution result of the address data to be resolved based on the target address code includes: Processing the target address encoding using a preset pointer network to obtain an initial parsing result of the address data to be parsed; wherein the initial parsing result includes all optional entities in the address data to be parsed; Determine whether there is entity nesting among all the optional entities; wherein entity nesting indicates that there are overlapping characters between entities; If so, the initial parsing result is processed based on a preset rule to obtain the target parsing result; The preset rules include: If two adjacent entities are of the same entity type, the entity with the first character appearing earlier is retained; If two adjacent entities are of different entity types, the end position of the first entity is corrected to be the position before the start position of the second entity; wherein, the first entity is the entity whose first character appears relatively earlier in the order of the two adjacent entities; and the second entity is the entity whose first character appears later in the order of the first entity.

8. An address element parsing device, characterized in that: include: A first acquisition module, used to acquire address data to be parsed; The encoding module is used to perform feature encoding on the address data to be parsed using a preset Lebert model to obtain a target address code; wherein the attention calculation in the preset Lebert model adopts a non-parametric relative position encoding; the attention calculation in the preset Lebert model adopts the following formula: ,in, The query vector representing the i-th element in the address data, represents the feature obtained after the i-th element is mapped by the embedding layer in the preset Lebert model, represents the first preset parameter matrix, represents the second preset parameter matrix, represents the third preset parameter matrix, It represents the feature obtained after the jth element in the address data is mapped by the embedding layer in the preset Lebert model. represents the key vector of the j-th element, represents the value vector of the j-th element, represents the attention score of the i-th element and the j-th element, Represents the feature vector of the i-th element after attention, represents the relative position code of the i-th element and the j-th element, and Calculate using the following formula: , where k represents the index of the kth position in the hidden vector, represents the dimension of the hidden vector; A determination module is used to determine the target parsing result of the address data to be parsed based on the target address code; wherein the target parsing result includes: the starting position of each entity, the ending position of each entity and the type of each entity, and the entity represents a word segmentation result with independent semantics in the address data to be parsed.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images