An address unit resolution method and system based on two-stage classification and labeling

Abstract

The application discloses a kind of address unit resolution method and device based on two-stage classification and labeling, belong to address unit resolution field;The specific steps of the method are as follows: S1 pre-processes training set;S2 trains address unit existence determination model;S3 trains address unit labeling model;S4 uses model to carry out address unit resolution;The present application is refined by two stages, and limited resources are used for the text that may exist address unit and the address unit labeling model with higher accuracy, which can improve the resolution efficiency of given type address unit.

Description

Technical Field

[0001] This invention discloses an address unit parsing method and system based on two-stage classification and annotation, which relates to the field of address unit parsing technology. Background Technology

[0002] Address unit parsing extracts various elements describing spatial location from texts such as express delivery slips, insurance policies, work orders, invoices, and contracts. These elements include: province, city, county, township, village, road, building, house number, direction, and distance. It has significant application value in industries such as logistics, express delivery, e-commerce, and trade. For example, parsing the sentence "We drove to Jiafushun Restaurant located at No. 85 Jiangwan North Road, Lingxi Town, Cangnan County, Wenzhou City" yields the following results: "City = Wenzhou City", "County = Cangnan County", "Town = Lingxi Town", "Road = Jiangwan North Road", "Road No. = 85", and "Poi = Jiafushun Restaurant".

[0003] Common address unit parsing methods include: keyword matching, dictionary matching, address tree matching, sequence labeling, deep learning, and a combination of deep learning and sequence labeling. The first three methods are matching-based, differing only in the elements involved in the matching and the data structure of the address units. These methods lack learning capabilities and cannot identify unknown address units. Method 4 is based on language models and probabilistic statistics, method 5 is based on deep learning temporal networks, and method 6 is a combination of methods 4 and 5. These three methods possess reasoning capabilities and can identify unknown address units, but their computational complexity is relatively high, especially in scenarios with numerous address unit types (10 or more) and long texts.

[0004] Furthermore, some texts contain nested address units (one or more other address units within a single address unit). For example, in the sentence "Zhang San lives near the Keqiao District Archives Bureau located on Huaqi Road," "Keqiao District Archives Bureau" is an address unit, and "Keqiao District" within it is also an address unit; this is called a nested address unit. For commonly used address unit parsing methods, the first three methods can identify nested address units through matching, but their scalability is limited because they cannot identify unknown address units. The latter three methods are model training methods based on labeled data; each address unit has only one annotation, making it impossible to annotate nested address units within it, thus failing to identify nested address units.

[0005] Therefore, an address unit resolution method and system based on two-stage classification and labeling is invented to solve the above problems. Summary of the Invention

[0006] This invention addresses the problems of existing technologies by providing a method and system for address unit resolution based on two-stage classification and annotation. The technical solution adopted is as follows: a method for address unit resolution based on two-stage classification and annotation, the specific steps of which are as follows:

[0007] S1 preprocesses the training set;

[0008] S2 training address cell existence determination model;

[0009] S3 training address cell annotation model;

[0010] S4 uses a model for address unit resolution.

[0011] The specific steps of S1 in preprocessing the training set are as follows:

[0012] S11 generates a text classification dataset;

[0013] S12 generates a sequence labeling dataset;

[0014] S13 returns the two generated datasets.

[0015] The specific steps of the S2 training address unit existence determination model are as follows:

[0016] S21 Initializes the address unit type number i = 1;

[0017] S22 Generate Dataset EAS text vectorization dataset str and address unit type label dataset

[0018] S23 sets the hyperparameters of the address unit existence determination model;

[0019] S24 sets the class weight parameters for the Focal loss function;

[0020] S25 initializes the parameters of the address unit existence determination model;

[0021] S26 will vectorize the text dataset EAS str Existence determination model for input address units The input terminal;

[0022] S27 will include the address unit type label dataset. Existence determination model for input address units The output terminal;

[0023] S28 trains the address cell existence determination model based on convergence conditions.

[0024] S29 sets i = i + 1;

[0025] If i≤N, then go to S2A; otherwise, go to S2B.

[0026] S2B returns the existence determination model of the generated address unit.

[0027] The specific steps for training the S3 address unit annotation model are as follows:

[0028] S31 initializes the address unit type number i = 1;

[0029] S32 generates dataset Text vectorization dataset and labeled sequence vectorization dataset

[0030] S33 sets the hyperparameters of the address cell annotation model;

[0031] S34 initializes the parameters of the address unit annotation model;

[0032] S35 will vectorize the text dataset. Input address unit annotation model The input terminal;

[0033] S36 will vectorize the labeled sequence dataset. Input address unit annotation model The output terminal;

[0034] S37 trains the address cell annotation model based on convergence conditions.

[0035] S38 sets i = i + 1;

[0036] S39 If i≤N, then go to S32; otherwise go to S3A.

[0037] S3A returns the generated address unit annotation model.

[0038] The specific steps of S4 in using the model to resolve address units are as follows:

[0039] S41 Initializes the address unit set UAS test If empty, the address unit type number i = 1;

[0040] S42 follows the given length Len cls and rule truncation Str test , obtain the text Str test,Len ;

[0041] S43 uses a vectorized model to process text. test,Len The word vector sequence is obtained.

[0042] S44 will use word vector sequences Existence determination model for input address units The input terminal;

[0043] S45 Operating Address Unit Existence Determination Model The probability of the existence of the address unit is obtained from the output.

[0044] S46 If Then switch to S47; otherwise switch to S4D.

[0045] S47 uses a vectorized model to process text. test The word vector sequence is obtained.

[0046] S48 will use word vector sequences Input address unit annotation model The input terminal;

[0047] S49 Running Address Unit Annotation Model Obtain the vectorized sequence of address unit tags from the output.

[0048] S4A is based on the vectorized sequence Determine the text Str test Corresponding label sequence

[0049] S4B based on the labeled sequence From the text Str test Extracting the address unit UA test,i ;

[0050] S4C will use address unit UA test,i Add to address unit set UAS test middle;

[0051] S4D sets i = i + 1;

[0052] If i≤N, then go to S44; otherwise go to S4F.

[0053] S4F returns the generated set of address units, UAS. test .

[0054] An address unit resolution system based on two-stage classification and labeling, the system specifically includes:

[0055] The training set preprocessing component M1 is used to convert the training set into a text classification dataset and a sequence labeling dataset;

[0056] Address cell existence determination model training component M2 is used to train a binary classification model to determine whether the text to be processed contains address cells of a given type.

[0057] Address cell labeling model training component M3 is used to train a sequence labeling model that calculates the location information of a given type of address cell;

[0058] The address unit parsing component M4 is used to extract address units from the text to be processed based on the address unit existence determination model and the address unit annotation model.

[0059] The training set preprocessing component M1 includes:

[0060] The text classification dataset generation component M11 is used to generate a dataset consisting of text and its address unit type labels;

[0061] The sequence labeling dataset generation component M12 is used to generate a dataset of single-type address unit labeling information.

[0062] The address unit existence determination model training component M2 includes:

[0063] The classification model input / output data generation component M21 is used to generate corresponding text vectorization datasets and address unit type label datasets based on the text classification dataset.

[0064] The classification model hyperparameter setting component M22 is used to set the type and structural parameters of the address cell existence determination model;

[0065] Focal loss function category weight setting component M23 is used to set the type weights of the Focal loss function according to the proportion of text containing different types of address units;

[0066] The classification model parameter initialization component M24 is used to initialize the weight matrices and bias vectors of different modules of the address cell existence determination model;

[0067] The classification model input terminal setting component M25 is used to feed the text vectorized dataset into the input terminal of the address cell existence determination model;

[0068] The classification model output setting component M26 is used to sequentially feed the address cell type label dataset into the output of the address cell existence determination model.

[0069] The classification model training convergence component M27 is used to train different modules of the address cell existence determination model according to the convergence conditions.

[0070] The address unit annotation model training component M3 includes:

[0071] The annotation model input / output data generation component M31 generates corresponding text vectorized datasets and annotation sequence vectorized datasets based on the sequence annotation datasets;

[0072] The hyperparameter setting component M32 for the annotation model is used to set the type and structural parameters of the address unit annotation model;

[0073] The annotation model parameter initialization component M33 is used to initialize the weight matrix and bias vector of different modules of the address unit annotation model;

[0074] The annotation model input setting component M34 is used to feed the text vectorized dataset into the input of the address cell annotation model;

[0075] The annotation model output setting component M35 is used to sequentially feed the vectorized dataset of the annotation sequence into the output of the address unit annotation model;

[0076] The annotation model training convergence component M36 is used to train different modules of the address unit annotation model according to the convergence conditions.

[0077] The address unit parsing component M includes:

[0078] Address cell existence probability generation component M41 is used to generate address cell existence probability based on address cell existence determination model;

[0079] Address unit existence determination component M42 is used to determine whether the text to be processed has an address unit of the corresponding type based on the probability of the address unit's existence.

[0080] Address unit label vectorization sequence generation component M43 is used to generate address unit label vectorization sequence based on address unit labeling model;

[0081] Address unit label sequence generation component M44 is used to generate the address unit label sequence of the text to be processed based on the address unit label vectorization sequence;

[0082] Address cell extraction component M45 is used to extract address cells from the address cell annotation sequence of the text to be processed.

[0083] The beneficial effects of this invention are as follows: This invention employs a strategy of parsing different types of address units one by one, which reduces the computational complexity caused by parsing all types of address units simultaneously. This allows the model to focus on the parsing task of specific types of address units, avoiding consideration of the influence of other types of address units, thereby improving the accuracy of address unit parsing. Simultaneously, it can also parse nested address units belonging to different types. A two-stage strategy is used to improve the speed of parsing different types of address units. First, an address unit existence determination model is used to filter out text from which a given type of address unit cannot exist, thus avoiding a large number of invalid address unit annotation calculations. Then, an address unit annotation model is used to calculate and extract address units of the corresponding type. Through this two-stage refined processing, limited resources are used for text that may contain address units and for address unit annotation models with high accuracy, thereby improving the parsing efficiency of a given type of address unit. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0085] Figure 1 This is a flowchart of the address unit resolution method based on two-stage classification and annotation of the present invention; Figure 2 This is a flowchart showing the specific steps of step S1; Figure 3 This is a flowchart showing the specific steps of step S11; Figure 4 This is a flowchart detailing the steps in step S12; Figure 5 This is a flowchart showing the specific steps of step S2; Figure 6 This is a flowchart showing the specific steps of step S22; Figure 7 This is a flowchart detailing the steps in step S3; Figure 8 This is a flowchart showing the specific steps of step S32; Figure 9 This is a flowchart showing the specific steps of step S4; Figure 10 This is a block diagram of the address unit resolution system based on two-stage classification and labeling of the present invention. Detailed Implementation

[0086] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0087] Example 1:

[0088] A two-stage classification and annotation-based address unit resolution method is described below:

[0089] S1 preprocesses the training set;

[0090] S2 training address cell existence determination model;

[0091] S3 training address cell annotation model;

[0092] S4 uses a model for address unit resolution;

[0093] The training set DAS org Convert to text classification dataset and sequence labeling datasets N represents the total number of address unit types; denoted by A1, A2, ..., A... N Indicates N types of address units, further, such as Figure 2 As shown, step S1 specifically includes:

[0094] S11: Generate a text classification dataset;

[0095] This step generates a text classification dataset. This dataset records whether the text contains a type A. i The address unit, its text quantity and DAS org The number of address unit label sequences is the same; The elements in are represented as Where Str is DAS org The original text corresponding to the address unit annotation sequence in the middle. It is its address unit type label, if DAS org The corresponding address unit label sequence contains type A i The address unit, then otherwise Furthermore, such as Figure 3 As shown, step S11 specifically includes:

[0096] S111: Initialize the text classification dataset It is an empty set;

[0097] S112: Read the training set DAS org The first address unit label sequence;

[0098] S113: Read the text Str corresponding to the address unit annotation sequence;

[0099] S114: Initialize address unit type number i = 1;

[0100] S115: If the address unit label sequence contains type A i If the address unit is not found, then go to S116; otherwise go to S117.

[0101] S116: Settings Transfer to S118;

[0102] S117: Settings

[0103] S118: Will Add to dataset

[0104] S119: Set i = i + 1;

[0105] S11A: If i≤N, then go to S115; otherwise go to S11B.

[0106] S11B: If the training set DAS org If there is an unread address unit label sequence, then switch to S11C; otherwise, switch to S11D.

[0107] S11C: Read the training set DAS org The next address unit label sequence, go to S113;

[0108] S11D: Returns the generated text classification dataset.

[0109] S12: Generate sequence labeling dataset;

[0110] This step generates the sequence labeling dataset. The record type of this dataset is A. i Address unit labeling information, The elements in are represented as Among them, Str i It is DAS org Contains address unit type A i The original text of the labeled sequence, It is a record of type A i The address unit location information is labeled in sequence, and the BMESO method is used to label each word in sequence; where B indicates the start of an entity, M indicates the middle of an entity, E indicates the end of an entity, S indicates an entity composed of words, and O indicates that it is not an entity; further, as Figure 4 As shown, step S12 specifically includes:

[0111] S121: Initialize the sequence labeling dataset It is an empty set;

[0112] S122: Read the training set DASorg The first address unit label sequence;

[0113] S123: Read the text Str corresponding to the address unit annotation sequence;

[0114] S124: Initialize the address unit type number i = 1;

[0115] S125: If the address unit label sequence contains type A i If the address unit is not found, then go to S126; otherwise go to S129.

[0116] S126: Generate a record for record type A i The label sequence of address unit location information

[0117] This step uses the BMESO method to label type A. i Address unit location information; using Seq org This refers to the labeled sequence being read. Further, step S126 specifically includes:

[0118] S1261: Copy Seq org The annotation information to

[0119] S1262: Will The address unit type is not A i All individual characters are marked with O;

[0120] S1263: Read The first address unit is of type A i entity

[0121] S1264: If If the entity is composed of single characters, then convert to S1265; otherwise, convert to S1266.

[0122] S1265: Will The corresponding single character is marked as O, then converted to S1269;

[0123] S1266: Will The first and last words of the corresponding words are marked as B and E respectively;

[0124] S1267: If If the length of the corresponding word is greater than 2, then proceed to S1268; otherwise, proceed to S1269.

[0125] S1268: Will The middle character of each word is marked with M;

[0126] S1269: If There are unread addresses with address unit type A. i If the entity is correct, then proceed to S126A; otherwise, proceed to S126B.

[0127] S126A: Read The next address unit is of type A i entity Transfer to S1264;

[0128] S126B: Returns the generated labeled sequence

[0129] S127: Copy text Str to Str i ;

[0130] S128: Will Add to dataset

[0131] S129: Set i = i + 1;

[0132] S12A: If i≤N, then go to S125; otherwise go to S12B.

[0133] S12B: If the training set DAS org If there is an unread address unit label sequence, then switch to S12C; otherwise, switch to S12D.

[0134] S12C: Read the training set DAS org The next address unit label sequence, go to S123;

[0135] S12D: Returns the generated sequence label dataset.

[0136] S13: Returns the two generated datasets;

[0137] Furthermore, the S2 training is used to determine whether the text to be processed contains a type A. i Binary classification model of address units N represents the total number of address unit types; for the training set DAS org For example, if the type is A i If the number of address units is small, then the dataset middle The relatively small number of texts leads to an imbalanced data distribution, affecting classification performance. Therefore, a Focal loss function is needed to reduce the weights of a large number of easily classified texts, allowing the model to focus more on a small number of difficult-to-classify texts during training. Furthermore, as... Figure 5 As shown, step S2 specifically includes:

[0138] S21: Initialize the address unit type number i = 1;

[0139] S22: Generate dataset EAS text vectorization dataset str and address unit type label dataset

[0140] This step reads and processes the dataset. elements Obtain the text vectorized dataset EAS str and address unit type label dataset Furthermore, such as Figure 6 As shown, step S22 specifically includes:

[0141] S221: Initialize EAS str and It is an empty set;

[0142] S222: Read the dataset The first element

[0143] S223: Read the text string Str;

[0144] S224: According to the given length Len cls By truncating Str according to the rules, we obtain the text Str. Len ;

[0145] S225: Using a vectorized model to process text. Len The word vector sequence is obtained.

[0146] This step uses a vector model to generate the text Str Len The character vector of each word in the text is then processed according to their positions in the text string. Len By concatenating the elements in the correct order, we obtain the text Str. Len The word vector sequence;

[0147] S226: Will Added to the text vectorization dataset EAS str ;

[0148] S227: Will Add to address unit type label dataset

[0149] S228: If There are unread elements Then switch to S229; otherwise switch to S22A.

[0150] S229: Read The next element Transfer to S223;

[0151] S22A: Returns the generated text vectorized dataset EAS str and address unit type label dataset

[0152] S23: Set the hyperparameters of the address unit existence determination model;

[0153] This step sets the type and structure of the model to be trained. The type can be selected from SVM, CNN, LSTM, BERT, etc., and the structure includes the type of kernel function, the number of network layers, the number of neurons, etc.

[0154] S24: Set the class weight parameters for the Focal loss function;

[0155] In address unit classification model In Chinese, the Focal loss function is defined as follows:

[0156]

[0157] Where t is the address unit type label, satisfying γ i It is a focused index, alpha. i,t and p i,t It is the class weight and probability estimate of the address unit type label t, where class weight α i,t The possible values ​​are as follows:

[0158]

[0159] Where, α i It is to satisfy The text in the dataset The proportion held by;

[0160] S25: Initialize the parameters of the address unit existence determination model;

[0161] This step initializes the weight matrices and bias vectors of different modules of the model to be trained;

[0162] S26: Vectorize the text dataset EAS str Existence determination model for input address units The input terminal;

[0163] S27: Transfer the address unit type label dataset Existence determination model for input address units The output terminal;

[0164] S28: Train the address cell existence determination model based on the convergence condition.

[0165] S29: Set i = i + 1;

[0166] S2A: If i≤N, then go to S22; otherwise go to S2B.

[0167] S2B: Returns the existence determination model of the generated address unit.

[0168] Furthermore, the S3 training is used to compute type A i Sequence labeling model of address unit location information N represents the total number of address unit types; further, such as Figure 7 As shown, step S3 specifically includes:

[0169] S31: Initialize the address unit type number i = 1;

[0170] S32: Generate dataset Text vectorization dataset and labeled sequence vectorization dataset

[0171] This step reads and processes the dataset. elements Obtain the text vectorized dataset and labeled sequence vectorization dataset Furthermore, such as Figure 8 As shown, step S32 specifically includes:

[0172] S321: Initialization and It is an empty set;

[0173] S322: Read the dataset The first element

[0174] S323: Read text Str i ;

[0175] S324: Using a vectorized model to process text. i The word vector sequence is obtained.

[0176] This step uses a vector model to generate the text Str i The character vector of each word in the text is then processed according to their positions in the text string. i By concatenating the elements in the correct order, we obtain the text Str. i The word vector sequence;

[0177] S325: Will Add to text vectorization dataset

[0178] S326: Generate labeled sequences vectorized sequence

[0179] This step involves labeling the sequence. Each element is one-hot encoded, and the encoded result of each element is a vector of length 5, where only one bit is 1 and the remaining four bits are 0, used to represent the five labels of the BMESO annotation method, and then they are arranged according to their positions in the annotation sequence. By concatenating the elements in order, a labeled sequence is obtained. Vectorized sequence;

[0180] S327: Will Add to labeled sequence vectorization dataset

[0181] S328: If There are unread elements Then switch to S329; otherwise switch to S32A.

[0182] S329: Read The next element Switch to S323;

[0183] S32A: Returns the generated vectorized text dataset. and labeled sequence vectorization dataset

[0184] S33: Set the hyperparameters of the address cell annotation model;

[0185] This step sets the type and structure of the model to be trained. The type can be selected from CRF, RNN, LSTM, Bert, ERNIE, etc., and the structure includes the number of network layers, optimizer, learning rate, batch size, etc.

[0186] S34: Initialize the parameters of the address unit annotation model;

[0187] This step initializes the weight matrices and bias vectors of different modules of the model to be trained;

[0188] S35: Vectorizing the Text Dataset Input address unit annotation model The input terminal;

[0189] S36: Vectorize the labeled sequence dataset Input address unit annotation model The output terminal;

[0190] S37: Train the address cell annotation model based on the convergence condition.

[0191] S38: Set i = i + 1;

[0192] S39: If i≤N, then go to S32; otherwise go to S3A.

[0193] S3A: Returns the generated address cell annotation model

[0194] Furthermore, this step uses a trained model to extract address units from the text to be processed; using Str test Represents the text to be processed, using UAS test This represents the corresponding set of address units; further, such as Figure 9 As shown, step S4 specifically includes:

[0195] S41: Initialize the address unit set UAS test If empty, the address unit type number i = 1;

[0196] S42: According to the given length Len cls and rule truncation Str test , obtain the text Str test,Len ;

[0197] S43: Using a vectorized model to process text Obtain the word vector sequence

[0198] S44: Transform the word vector sequence Existence determination model for input address units The input terminal;

[0199] S45: Existence Determination Model for Running Address Units The probability of the existence of the address unit is obtained from the output.

[0200] S46: If Then switch to S47; otherwise switch to S4D.

[0201] S47: Using a vectorized model to process text. test The word vector sequence is obtained.

[0202] S48: Transform the word vector sequence Input address unit annotation model The input terminal;

[0203] S49: Runtime Address Unit Annotation Model Obtain the vectorized sequence of address unit tags from the output.

[0204] S4A: Based on the vectorized sequence Determine the text Str test Corresponding label sequence

[0205] S4B: Based on the labeled sequence From the text Str test Extracting the address unit UA test,i ;

[0206] S4C: Move address unit UA test,i Add to address unit set UAS test middle;

[0207] S4D: Set i = i + 1;

[0208] S4E: If i≤N, then go to S44; otherwise go to S4F.

[0209] S4F: Returns the generated set of address units (UAS) test .

[0210] Example 2:

[0211] Based on Example 1, let the training set DAS be set. org It contains 10 different types of address units: A1 = Provincial administrative region, A2 = Municipal administrative region, A3 = County administrative region, A4 = Township administrative region, A5 = Village administrative region, A6 = Road, A7 = Point of Interest, A8 = Number, A9 = Direction, A 10 = Distance; Example as follows (multiple address units of the same type separated by "|"):

[0212] Q1. We drove to No. 00, Jianshe Road West, Jianli County, Jingzhou City, Hubei Province;

[0213] A1 = Hubei Province, A2 = Jingzhou City, A3 = Jianli County, A6 = Jianshe Road, A8 = No. 00, A9 = West

[0214] Q2. The building located opposite Helong Middle School on Hongwei North Road is the Administrative Service Center of Honghu City Public Security Bureau.

[0215] A2 = Honghu City, A6 = Hongwei North Road, A7 = Helong Middle School | Honghu City Public Security Bureau Administrative Service Center, A9 = Opposite

[0216] Q3. The crime scene was located in a vegetable field 200 meters east of Shigaozhuang Bridge in Huilongguan Town, Changping District;

[0217] A3 = Changping District, A4 = Huilongguan Town, A5 = Shigaozhuang, A7 = Shigaozhuang Bridge, A9 = East, A 10 =200 meters

[0218] In Example Q2, "Honghu City" is a nested address unit located within "Honghu City Public Security Bureau Administrative Service Center"; in Example Q3, "Shigezhuang" is a nested address unit located within "Shigezhuang Bridge".

[0219] Training set DAS org It contains 10,000 data entries, and the number of data entries containing the above 10 types of addresses are: 3200, 4650, 4850, 3400, 2300, 2100, 3600, 2000, 1400, and 700 respectively.

[0220] Execute S11: Generate a text classification dataset These respectively indicate whether the text contains text of type A1, A2, ..., A 10 The address units and their corresponding address unit type labels are respectively Each dataset contains 10,000 text entries; for example Q3 above, it contains 6 entity classes: A3, A4, A5, A7, A9, A... 10 Therefore, the labels of this example in different datasets are respectively

[0221] Execute S12: Generate sequence labeling dataset They represent types A1, A2, ..., A, respectively. 10 The address unit annotation information, the number of sequence entries in each dataset are 3200, 4650, 4850, 3400, 2300, 2100, 3600, 2000, 1400, and 700 respectively; for example Q3 above, it contains 6 types of entities: A3, A4, A5, A7, A9, A 10 The processed results are as follows:

[0222] A and A3 correspond to the entity "Changping District", and the generated annotation sequence Add to dataset

[0223] B and A4 correspond to the entity "Huilongguan Town", and the generated annotation sequence Add to dataset

[0224] C and A5 correspond to the entity "Shigezhuang", and the generated annotation sequence Add to dataset

[0225] D and A7 correspond to the entity "Shigezhuang Bridge", and the generated annotation sequence Add to dataset

[0226] E and A9 correspond to the entity "East", and the generated annotation sequence Add to dataset

[0227] F, A 10 The generated annotation sequence corresponding to the entity "200 meters" Add to dataset

[0228] Execute S21: Initialize address unit type number i = 1;

[0229] Execution S22: First, use paddlenlp.transformers.BertTokenizer to load the basic Chinese Bert encoder for text vectorization, setting the parameters: number of encoding layers = 12, hidden state dimension = 768; for the dataset For each text Str, extract the length Len. cls =128; Str is generated by truncating from both ends of Str. Len This involves concatenating the first 64 characters and the last 64 characters of Str; if Str is less than 64 characters long, then padding the beginning and end of Str with "O" symbols to make it 64 characters long before concatenating; Str is processed one by one using the basic Chinese Bert encoder. Len The single word w in c This yields the encoding value of each character in the vocabulary, as well as the vector of each character across 12 encoding layers. Each vector has a dimension of 768. The average vector from the last four layers is used as the character vector. According to the single word w c In the text Str Len Sequential concatenation of word vectors Get the text Str Len word vector sequence All word vector sequences The resulting set is the dataset. EAS text vectorization dataset str ;

[0230] Then, from the dataset Read the address unit type label of each text Str one by one Then merge them together to get the dataset. Address unit type label dataset

[0231] Execution S23: Use paddlenlp.transformers.BertForSequenceClassification to load the basic Chinese BERT classifier for text classification, and set the parameters as follows: number of classes = 2, batch size = 32, optimizer = Adam, learning rate = 1e-6;

[0232] Execute S24: First, calculate the condition that satisfies The text in the dataset The proportions of these proportions are as follows: α1 = 0.32, α2 = 0.465, α3 = 0.485, α4 = 0.34, α5 = 0.23, α6 = 0.21, α7 = 0.36, α8 = 0.2, α9 = 0.14, α 10 =0.07;

[0233] Then, according to α i Calculate the address unit type label t in different datasets Category weight α i,t t = 0, 1, as shown in the table below:

[0234]

[0235]

[0236] Finally, set the focus index γ i =2, according to α i,t and γ i Establish a model for determining the existence of different address units Focal loss function;

[0237] Execute S25: Initialize the address cell existence determination model using the parameters of the pre-trained BERT classification model. The weight matrix and bias vector;

[0238] Perform S26 and S27: Vectorize the text dataset EAS respectively. str and address unit type label dataset Existence determination model for input address units The input and output terminals;

[0239] Execute S28: Iterate and train for several rounds according to the given parameters until the address cell existence determination model is reached. The F1 score on the validation set reaches its maximum value and then begins to decrease;

[0240] Execute S29: Set i = i + 1;

[0241] Execute S2A: If i≤10, then go to S22; otherwise go to S2B.

[0242] Execute S2B: Return the generated address unit existence determination model

[0243] Execute S31: Initialize address unit type number i = 1;

[0244] Execution S32: First, use paddlenlp.transformers.BertTokenizer to load the basic Chinese Bert encoder for text vectorization, setting the parameters: number of encoding layers = 12, hidden state dimension = 768; for the dataset Each text Str i Using the basic Chinese BERT encoder, each Str file is processed individually. i The single word w in i,t This yields the encoding value of each character in the vocabulary, as well as the vector of each character across 12 encoding layers. Each vector has a dimension of 768. The average vector from the last four layers is used as the character vector. According to the single word w i,t In the text Str i Sequential concatenation of word vectors Get the text Str i word vector sequence All word vector sequences The resulting set is the dataset. Text vectorization dataset

[0245] Then, from the dataset Read text one by one and text Str i Corresponding label sequence Each element in this sequence takes a value from the set {'B', 'M', 'E', 'S', 'O'}, representing the five labels of the BMESO annotation method, with ID codes 1, 2, 3, 4, and 5 respectively; Each element is one-hot encoded, and the encoded result of each element is a vector of length 5, where only one bit is 1 and the remaining four bits are 0. The dimension number with a value of 1 is the same as its ID code; according to their position in the labeled sequence... By concatenating the elements in order, a labeled sequence is obtained. vectorized sequence All vectorized sequences The resulting set is the dataset. Annotated sequence vectorization dataset

[0246] Execution S33: Use paddlenlp.transformers.BertForTokenClassification to load the basic Chinese Bert annotator for sequence labeling, and set the parameters as follows: number of entity types = 5, batch size = 64, optimizer = AdamW, learning rate = 2e-5, loss function = cross-entropy;

[0247] Execute S34: Initialize the address cell annotation model using the parameters of the pre-trained BERT annotation model. The weight matrix and bias vector;

[0248] Perform S35 and S36: Vectorize the text dataset respectively. and labeled sequence vectorization dataset Input address unit annotation model The input and output terminals;

[0249] Execute S37: Iterate through the training process for several rounds with the given parameters until the address cell labeling model is reached. The F1 score on the validation set reaches its maximum value and then begins to decrease;

[0250] Execute S38: Set i = i + 1;

[0251] Execute S39: If i≤10, then go to S32; otherwise go to S3A.

[0252] Execute S3A: Returns the generated address cell annotation model.

[0253] Execute S41: Let Str test =“The building located opposite Helong Middle School on Hongwei North Road is the Administrative Service Center of Honghu City Public Security Bureau;”, Initialize the address unit set UAS test =φ, address unit type number i = 1;

[0254] Execute S42: Set the truncation length Len cls =128, from Str testExtract from both ends, fill the empty spaces with the symbol "O", and generate Str. test,Len =“The building opposite Helong Middle School on Hongwei North Road is the Administrative Service Center of Honghu City Public Security Bureau;OO ...

[0255] Execution S43: The basic Chinese Bert encoder is loaded using paddlenlp.transformers.BertTokenizer for text vectorization, with the following parameters set: number of encoding layers = 12, hidden state dimension = 768; the basic Chinese Bert encoder is then used to process each Str... test,Len The single word w in test,Len,t This yields the encoding value of each character in the vocabulary, as well as the vector of each character across 12 encoding layers. Each vector has a dimension of 768. The average vector from the last four layers is used as the character vector. According to the single word w test,Len,t In the text Str test,Len Sequential concatenation of word vectors Get the text Str test,Len word vector sequence The sequence has a length of 128, and each element is a vector with a dimension of 768;

[0256] Execute S44: Convert the word vector sequence Existence determination model for input address units The input terminal;

[0257] Execution S45: Running Address Unit Existence Determination Model The probability of the existence of the address unit is obtained from the output.

[0258] Execute S46: When i = 2, 6, 7, 9, the following condition is met. Switch to S47; otherwise switch to S4D.

[0259] Execution S47: Use paddlenlp.transformers.BertTokenizer to load the basic Chinese Bert encoder for text vectorization, setting the parameters: number of encoding layers = 12, hidden state dimension = 768; use the basic Chinese Bert encoder to process each Str... test The single word w in test,tThis yields the encoding value of each character in the vocabulary, as well as the vector of each character across 12 encoding layers. Each vector has a dimension of 768. The average vector from the last four layers is used as the character vector. According to the single word w test,t In the text Str test Sequential concatenation of word vectors Get the text Str test word vector sequence The sequence has a length of 30, and each element is a vector with a dimension of 768;

[0260] Execute S48: Convert the word vector sequence Input address unit annotation model The input terminal;

[0261] Execute S49: Run the address unit annotation model Obtain the vectorized sequence from the output. The sequence is 30 in length, and each element is a 5-dimensional vector representing Str. test The probability that the corresponding single-character label in the text is 'B', 'M', 'E', 'S', or 'O'.

[0262] Execute S4A: For For each element, the dimension number of its largest component is Str. test The ID code of the corresponding single-word tag in the text Strain is used to determine the corresponding single-word tag, according to their position in the text Strain. test By concatenating the sequences in order, the corresponding labeled sequence can be obtained. When i = 2, 6, 7, 9, the corresponding labeling sequences are as follows:

[0263] A. When i = 2, label the sequence.

[0264] B. When i = 6, label the sequence.

[0265] C. When i = 7, label the sequence.

[0266] D. When i = 9, label the sequence.

[0267] Execute S4B: For labeled sequences For example, if the label "S" exists, then extract the text Str. testThe corresponding single word is used as the address unit; if the label "B" exists, then extract the label between "B" and its successor "E" in the text Str. test The corresponding set of single words is used as the address unit; when i = 2, 6, 7, 9, the corresponding address units are as follows (multiple address units of the same type are separated by "|"):

[0268] A. When i = 2, the address unit UA test,2 = "Honghu City";

[0269] B. When i = 6, the address unit UA test,6 = "Hongwei North Road";

[0270] C. When i = 7, the address unit UA test,7 = "Helong Middle School | Honghu City Public Security Bureau Administrative Service Center";

[0271] D. When i = 9, the address unit UA test,9 = "opposite";

[0272] Execute S4C: Move address unit UA test,i Add to address unit set UAS test middle;

[0273] Execute S4D: Set i = i + 1;

[0274] Execute S4E: If i≤10, then go to S44; otherwise go to S4F.

[0275] Execute S4F: Returns the generated set of address units (UAS) test = {"Honghu City", "Hongwei North Road", "Helong Middle School", "Honghu City Public Security Bureau Administrative Service Center", "Opposite"}.

[0276] Example 3:

[0277] This invention also provides an address unit resolution system based on two-stage classification and annotation, such as... Figure 10 As shown, it includes:

[0278] Training set preprocessing component M1 is used to convert the training set into a text classification dataset and a sequence labeling dataset. The training set preprocessing component M1 includes:

[0279] The text classification dataset generation component M11 is used to generate a dataset consisting of text and its address unit type labels;

[0280] The sequence labeling dataset generation component M12 is used to generate a dataset of single-type address unit labeling information;

[0281] Address cell existence determination model training component M2 is used to train a binary classification model to determine whether a given type of address cell exists in the text to be processed. The address cell existence determination model training component M2 includes:

[0282] The classification model input / output data generation component M21 is used to generate corresponding text vectorization datasets and address unit type label datasets based on the text classification dataset.

[0283] The classification model hyperparameter setting component M22 is used to set the type and structural parameters of the address cell existence determination model;

[0284] Focal loss function category weight setting component M23 is used to set the type weights of the Focal loss function according to the proportion of text containing different types of address units;

[0285] The classification model parameter initialization component M24 is used to initialize the weight matrices and bias vectors of different modules of the address cell existence determination model;

[0286] The classification model input terminal setting component M25 is used to feed the text vectorized dataset into the input terminal of the address cell existence determination model;

[0287] The classification model output setting component M26 is used to sequentially feed the address cell type label dataset into the output of the address cell existence determination model.

[0288] The classification model training convergence component M27 is used to train different modules of the address cell existence determination model according to the convergence conditions.

[0289] Address unit annotation model training component M3 is used to train a sequence annotation model that calculates the location information of address units of a given type. The address unit annotation model training component M3 includes:

[0290] The annotation model input / output data generation component M31 generates corresponding text vectorized datasets and annotation sequence vectorized datasets based on the sequence annotation datasets;

[0291] The hyperparameter setting component M32 for the annotation model is used to set the type and structural parameters of the address unit annotation model;

[0292] The annotation model parameter initialization component M33 is used to initialize the weight matrix and bias vector of different modules of the address unit annotation model;

[0293] The annotation model input setting component M34 is used to feed the text vectorized dataset into the input of the address cell annotation model;

[0294] The annotation model output setting component M35 is used to sequentially feed the vectorized dataset of the annotation sequence into the output of the address unit annotation model;

[0295] The annotation model training convergence component M36 is used to train different modules of the address unit annotation model according to the convergence conditions.

[0296] Address unit parsing component M4 is used to extract address units from the text to be processed based on the address unit existence determination model and the address unit annotation model. The address unit parsing component M4 includes:

[0297] Address cell existence probability generation component M41 is used to generate address cell existence probability based on address cell existence determination model;

[0298] Address unit existence determination component M42 is used to determine whether the text to be processed has an address unit of the corresponding type based on the probability of the address unit's existence.

[0299] Address unit label vectorization sequence generation component M43 is used to generate address unit label vectorization sequence based on address unit labeling model;

[0300] Address unit label sequence generation component M44 is used to generate the address unit label sequence of the text to be processed based on the address unit label vectorization sequence;

[0301] Address cell extraction component M45 is used to extract address cells from the address cell annotation sequence of the text to be processed.

[0302] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A two-stage classification and annotation-based address unit resolution method, characterized by: The specific steps of the method are as follows: S1 preprocesses the training set; S2 training address cell existence determination model; S3 training address cell annotation model; S4 uses a model for address unit resolution; The specific steps of S1 in preprocessing the training set are as follows: S11 generates a text classification dataset; The generated text classification dataset Does the dataset record text containing the type ? The address unit, its text quantity and The number of address unit label sequences is the same; The elements in are represented as ,in, yes The original text corresponding to the address unit annotation sequence in the middle. It is its address unit type label, if The corresponding address unit label sequence contains type The address unit, then ,otherwise Furthermore, step S11 specifically includes: S111: Initialize the text classification dataset , ... It is an empty set; S112: Read the training set The first address unit label sequence; S113: Read the text corresponding to the address unit annotation sequence. ; S114: Initialize address unit type number ; S115: If the address unit label sequence contains type... If the address unit is not found, then go to S116; otherwise go to S117. S116: Settings Turn onto S118; S117: Settings ; S118: Will Add to dataset ; S119: Settings ; S11A: If If yes, then switch to S115; otherwise, switch to S11B. S11B: If the training set If there is an unread address unit label sequence, then switch to S11C; otherwise, switch to S11D. S11C: Read the training set The next address unit label sequence, go to S113; S11D: Returns the generated text classification dataset. , ... ; S12 generates a sequence labeling dataset; The generated sequence labeling dataset The record type of this dataset is Address unit labeling information, The elements in are represented as ;in, yes Contains address unit type The original text of the labeled sequence, The record type is The address unit location information is labeled in sequence, and each word is labeled using the BMESO method; where B represents the start of an entity, M represents the middle of an entity, E represents the end of an entity, S represents an entity composed of words, and O represents no entity; further, step S12 specifically includes: S121: Initialize the sequence labeling dataset , ... It is an empty set; S122: Read the training set The first address unit label sequence; S123: Read the text corresponding to the address unit annotation sequence. ; S124: Initialize address unit type number ; S125: If the address unit label sequence contains type... If the address unit is not found, then go to S126; otherwise go to S129. S126: Generate data for record type... The label sequence of address unit location information ; Step S12 uses the BMESO method to annotate the type as follows: Address unit location information; using This refers to the read labeled sequence. Further, step S126 specifically includes: S1261: Copy The annotation information to ; S1262: Will The address unit type is not All individual characters are marked with O; S1263: Read The first address unit type is entity ; S1264: If If the entity is composed of single characters, then convert to S1265; otherwise, convert to S1266. S1265: Will The corresponding single character is labeled S, then convert it to S1269; S1266: Will The first and last words of the corresponding words are marked as B and E respectively; S1267: If If the length of the corresponding word is greater than 2, then proceed to S1268; otherwise, proceed to S1269. S1268: Will The middle character of each word is marked with M; S1269: If There are unread addresses with the address unit type being If the entity is correct, then proceed to S126A; otherwise, proceed to S126B. S126A: Read The next address unit type is entity Transfer to S1264; S126B: Returns the generated labeled sequence ; S127: Copy Text arrive ; S128: Will Add to dataset ; S129: Settings ; S12A: If If yes, switch to S125; otherwise, switch to S12B. S12B: If the training set If there is an unread address unit label sequence, then switch to S12C; otherwise, switch to S12D. S12C: Read the training set The next address unit label sequence, go to S123; S12D: Returns the generated sequence label dataset. , ... ; S13 returns the two generated datasets; The specific steps of the S2 training address unit existence determination model are as follows: S21 Initialization Address Unit Type Number ; S22 Generate Dataset Text vectorization dataset and address unit type label dataset ; S23 sets the hyperparameters of the address unit existence determination model; S24 sets the class weight parameters for the Focal loss function; S25 initializes the parameters of the address unit existence determination model; S26 will vectorize the text dataset. Existence determination model for input address units The input terminal; S27 will include the address unit type label dataset. Existence determination model for input address units The output terminal; S28 trains the address cell existence determination model based on convergence conditions. ; S29 Settings ; S2A if If yes, switch to S22; otherwise, switch to S2B. S2B returns the existence determination model of the generated address unit. , ; The specific steps for training the S3 address unit annotation model are as follows: S31 Initialization Address Unit Type Number ; S32 generates dataset Text vectorization dataset and labeled sequence vectorization dataset ; S32 reads and processes the dataset. elements To obtain the text vectorized dataset and labeled sequence vectorization dataset Furthermore, step S32 specifically includes: S321: Initialization and It is an empty set; S322: Read the dataset The first element ; S323: Read text ; S324: Using a vectorized model to process text The word vector sequence is obtained. ; This step uses a vectorization model to generate text. The character vector of each word in the text is then processed according to their positions within the text. By concatenating the elements in the correct order, we obtain the text. The word vector sequence; S325: Will Add to text vectorization dataset ; S326: Generate labeled sequences vectorized sequence ; This step involves labeling the sequence. Each element is one-hot encoded, and the encoded result of each element is a vector of length 5, where only one bit is 1 and the remaining four bits are 0, used to represent the five labels of the BMESO annotation method, and then they are arranged according to their position in the annotation sequence. By concatenating the elements in order, a labeled sequence is obtained. Vectorized sequence; S327: Will Add to labeled sequence vectorization dataset ; S328: If There are unread elements If yes, switch to S329; otherwise, switch to S32A. S329: Read The next element Turn to S323; S32A: Returns the generated vectorized text dataset. and labeled sequence vectorization dataset ; S33 sets the hyperparameters of the address cell annotation model; S34 initializes the parameters of the address unit annotation model; S35 will vectorize the text dataset. Input address unit annotation model The input terminal; S36 will vectorize the labeled sequence dataset. Input address unit annotation model The output terminal; S37 trains the address cell annotation model based on convergence conditions. ; S38 Settings ; S39 if If yes, switch to S32; otherwise, switch to S3A. S3A returns the generated address unit annotation model. , ; The specific steps of S4 in using the model to resolve address units are as follows: S41 Initialize Address Unit Set Empty, address unit type number ; S42 according to the given length and rule truncation , obtain text ; S43 uses a vectorized model to process text. The word vector sequence is obtained. ; S44 will use word vector sequences Existence determination model for input address units The input terminal; S45 Operating Address Unit Existence Determination Model Obtain the probability of the existence of the address unit from the output terminal. ; S46 If If yes, switch to S47; otherwise, switch to S4D. S47 uses a vectorization model to process text. The word vector sequence is obtained. ; S48 will use word vector sequences Input address unit annotation model The input terminal; S49 Running Address Unit Annotation Model Obtain the vectorized sequence of address unit labels from the output. ; S4A is based on the vectorized sequence Determine text Corresponding label sequence ; S4B based on the labeled sequence From the text Extracting address units ; S4C will address unit Add to address unit set middle; S4D Settings ; S4E if If yes, switch to S44; otherwise, switch to S4F. S4F returns the generated set of address units. .

2. A system using the address unit resolution method based on two-stage classification and annotation as described in claim 1, characterized in that: The system specifically includes: The training set preprocessing component M1 is used to convert the training set into a text classification dataset and a sequence labeling dataset; Address cell existence determination model training component M2 is used to train a binary classification model to determine whether the text to be processed contains address cells of a given type. Address cell labeling model training component M3 is used to train a sequence labeling model that calculates the location information of a given type of address cell; Address unit parsing component M4 is used to extract address units from the text to be processed based on the address unit existence determination model and the address unit annotation model. The training set preprocessing component M1 includes: The text classification dataset generation component M11 is used to generate a dataset consisting of text and its address unit type labels; The sequence labeling dataset generation component M12 is used to generate a dataset of single-type address unit labeling information; The address unit existence determination model training component M2 includes: The classification model input / output data generation component M21 is used to generate corresponding text vectorization datasets and address unit type label datasets based on the text classification dataset. The classification model hyperparameter setting component M22 is used to set the type and structural parameters of the address cell existence determination model; Focal loss function category weight setting component M23 is used to set the type weights of the Focal loss function according to the proportion of text containing different types of address units; The classification model parameter initialization component M24 is used to initialize the weight matrices and bias vectors of different modules of the address cell existence determination model; The classification model input terminal setting component M25 is used to feed the text vectorized dataset into the input terminal of the address cell existence determination model; The classification model output setting component M26 is used to sequentially feed the address cell type label dataset into the output of the address cell existence determination model. The classification model training convergence component M27 is used to train different modules of the address cell existence determination model according to the convergence conditions. The address unit annotation model training component M3 includes: The annotation model input / output data generation component M31 is used to generate corresponding text vectorized datasets and annotation sequence vectorized datasets based on the sequence annotation dataset; The hyperparameter setting component M32 for the annotation model is used to set the type and structural parameters of the address unit annotation model; The annotation model parameter initialization component M33 is used to initialize the weight matrix and bias vector of different modules of the address unit annotation model; The annotation model input setting component M34 is used to feed the text vectorized dataset into the input of the address cell annotation model; The annotation model output setting component M35 is used to sequentially feed the vectorized dataset of the annotation sequence into the output of the address unit annotation model; The annotation model training convergence component M36 is used to train different modules of the address unit annotation model according to the convergence conditions. The address unit parsing component M4 includes: Address cell existence probability generation component M41 is used to generate address cell existence probability based on address cell existence determination model; Address unit existence determination component M42 is used to determine whether the text to be processed has an address unit of the corresponding type based on the probability of the address unit's existence. Address unit label vectorization sequence generation component M43 is used to generate address unit label vectorization sequence based on address unit labeling model; Address unit label sequence generation component M44 is used to generate the address unit label sequence of the text to be processed based on the address unit label vectorization sequence; Address cell extraction component M45 is used to extract address cells from the address cell annotation sequence of the text to be processed.

Images