A method and apparatus for resolving address units

Abstract

This invention relates to the fields of voting fusion and address unit parsing, specifically providing an address unit parsing method and apparatus, comprising the following steps: S1, training multiple address unit annotation models; S2, processing data using multiple address unit annotation models; S3, fusing the processing results of multiple address unit annotation models; S4, extracting address unit annotation information from the fusion result. The theoretical basis of this invention is the fusion of annotation results using both absolute voting and maximum score methods. Compared with existing technologies, this invention solves the problem of the absolute voting method's inability to fuse data when the number of votes for all address units does not exceed half, and also addresses the significant bias and low reliability issues of the maximum score method when data types are unevenly distributed.

Description

Technical Field

[0001] This invention relates to the fields of voting fusion and address unit resolution, and specifically provides an address unit resolution method and apparatus. Background Technology

[0002] Address unit parsing is used to extract various geographical location elements such as province, city, county, township, village, road, building, house number, direction, and distance from texts such as express delivery slips, work orders, invoices, ID cards, and contracts. It has significant application value in industries such as express delivery, trade, e-commerce, and logistics. For example, parsing the sentence "We visited Qiaoyuan Farm in Xiaqiao, Sixi Town, Taishun County, Wenzhou City today" yields the results: "City = Wenzhou City", "County = Taishun County", "Town = Sixi Town", "Village = Xiaqiao", and "POI = Qiaoyuan Farm". Here, City, County, Town, Village, and Poi represent city-level administrative region, county-level administrative region, township-level administrative region, village-level administrative region, and point of interest, respectively.

[0003] Common address unit parsing methods include keyword matching, dictionary matching, address tree matching, sequence labeling, deep learning, and combinations thereof. Each method employs different data structures and processing flows, resulting in multiple implementation schemes. Training models using the same implementation scheme on different datasets exhibits significant bias due to differences in data features and distributions. On test sets with characteristics and distributions significantly different from the original training data, the model's inference performance degrades markedly.

[0004] Ensemble learning trains and merges multiple models for prediction. Its core idea is that even if one model makes a wrong prediction, the others can correct it. Therefore, in address unit resolution, ensemble learning eliminates the differences in inference performance distribution caused by model bias, resulting in a model with a balanced inference performance distribution, thus improving the model's resolution quality and expanding its applicability.

[0005] Voting is a commonly used ensemble learning method. In address unit resolution, this method considers the address type that appears most frequently in all model predictions as the fused address type. There are two types of voting: absolute voting and relative voting. Absolute voting considers the address type that appears more than half the time as the fused address type; if no address type appears more than half the time, the prediction is rejected. Relative voting considers the address type that appears most frequently as the fused address type. Comparatively, absolute voting receives more support votes than relative voting, indicating higher reliability; however, absolute voting may fail to fuse all addresses. The predicted scores for different address types represent the reliability of the test sample belonging to the corresponding address type. The maximum score method considers the address type with the highest score as the fused address type; however, if the data type distribution is unbalanced, the model shows obvious bias, and the prediction reliability of this method is not high. Summary of the Invention

[0006] This invention addresses the shortcomings of the prior art by providing a highly practical and reliable address unit resolution method.

[0007] A further technical objective of this invention is to provide a reasonably designed address unit parsing device.

[0008] The technical solution adopted by this invention to solve its technical problem is:

[0009] An address unit resolution method has the following steps:

[0010] S1. Train multiple address unit annotation models;

[0011] S2. Data is processed using a multiple address unit annotation model;

[0012] S3. The processing results of integrating multiple address unit annotation models;

[0013] S4. Extract address unit annotation information from the fusion results.

[0014] Furthermore, in step S1, the training set ADS is used. train Training a model with M address units i , 1≤i≤M.

[0015] Furthermore, in step S2, the trained address units are used to label the Model. i Processing test set ADS test The text in the range is 1≤i≤M;

[0016] For ADS test AddItem, Model, any address unit of the text Str to be processed iThe processing result is represented as (Tag) i Score i ), where Tag i and Score i They are using Model i The address type and score obtained after reasoning, for entities other than address units, are represented by the Tag. i =O, Score i =-1.

[0017] Furthermore, in step S3, the absolute voting method and the maximum score method are combined to merge the test set ADS. test M processing results (Tags) of any address unit AddItem in the middle i Score i ), 1≤i≤M;

[0018] Use (Tag) meg Score meg The result is represented by ), and |x| represents the rounding operation on x. Count represents the result. i Tag indicates address type i The number of occurrences is represented by k, which indicates the address type number that occurs most frequently, and t, which indicates the address type number with the highest score.

[0019] Furthermore, step S3 specifically includes:

[0020] S31, Read ADS test The first text to be processed in the middle, Str;

[0021] S32. Read the first address unit AddItem in Str;

[0022] S33. Read the M processing results (Tags) of AddItem. i Score i ), 1≤i≤M;

[0023] S34, Calculate Tag i Count the number of times it appears in M ​​address types i , 1≤i≤M;

[0024] S35, Calculate Count i maximum value

[0025] S36, if Then proceed to step S37; otherwise, proceed to step S3A.

[0026] S37. Calculate the address type number that appears most frequently.

[0027] S38. Determine the address type Tag after merging. meg =Tag k ;

[0028] S39. Determine the score after fusion based on the principle of the highest frequency of occurrence. Transform into S3D steps;

[0029] S3A: Address type number with the highest score

[0030] S3B: Determine the address type Tag after merging. meg =Tag t ;

[0031] S3C: Determine the merged score based on the principle of maximizing the total score.

[0032] S3D: Determine the fusion result of AddItem (Tag) meg Score meg );

[0033] S3E: If there are still unprocessed address units AddItem in Str, then proceed to step S3F; otherwise, proceed to step S3G.

[0034] S3F: Read the next address unit AddItem in Str, and proceed to step S33;

[0035] S3G: If ADS test If there is still unprocessed text to be processed, proceed to step S3H; otherwise, proceed to step S3I.

[0036] S3H: Read ADS test Next, proceed to step S32 for the text Str to be processed.

[0037] S3I: Return to test set ADS test The result of fusing all address units in the data.

[0038] Furthermore, in step S4, based on the fusion result, the address unit entity, address type, and score are extracted from the text to be processed.

[0039] An address unit parsing device includes an address unit annotation model training component, an address type and score generation component, an address type and score fusion component, and an address unit annotation information extraction component;

[0040] The address unit annotation model training component is used to train multiple address unit annotation models;

[0041] The address type and score generation component is used to obtain multiple address types and scores for each address unit;

[0042] The address type and score fusion component is used to fuse multiple address types and scores for each address unit;

[0043] The address unit annotation information extraction component is used to extract the address unit entity, address type, and score from the fusion result.

[0044] Furthermore, the address type and score fusion component includes an absolute voting method fusion sub-component, a maximum score method fusion sub-component, a fusion method selection sub-component, and a fusion result generation sub-component;

[0045] The absolute voting method fusion sub-component is used to fuse multiple address types and scores for each address unit using a comprehensive voting method;

[0046] The maximum score method fusion sub-component is used to fuse multiple address types and scores for each address unit using the maximum score method;

[0047] The fusion method selection sub-component is used to select the fusion address type and score based on the voting results.

[0048] The fusion result generation sub-component is used to generate the fusion result for each address unit based on the fused address type and score.

[0049] Compared with the prior art, the address unit resolution method and apparatus of the present invention have the following outstanding advantages:

[0050] This invention first counts the number of votes for all address types. If the maximum number of votes exceeds half of the total number of votes, then the absolute voting method is used for fusion. Otherwise, it indicates that the bias of the prediction result is relatively weak, so the maximum score method is used for fusion.

[0051] By combining the advantages of both methods, this invention solves the problem of the absolute voting method being unable to be integrated when the number of votes for all address units does not exceed half, and also solves the problem of the maximum score method having obvious bias and low reliability when the data type distribution is uneven. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] AppendixFigure 1 This is a flowchart illustrating an address unit resolution method;

[0054] Appendix Figure 2 This is a flowchart illustrating the processing results of multiple address unit annotation models in an address unit resolution method.

[0055] Appendix Figure 3 This is a block diagram of an address unit resolution device. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The following is a preferred embodiment:

[0058] like Figure 1-2 As shown, in this embodiment, an address unit resolution method is proposed, assuming a training set ADS. train The test set ADS contains 5000 labeled address texts. test Contains 1000 unlabeled address texts, ADS train and ADS test The address units in the middle are of the same type, as follows: Prov = provincial administrative region, City = municipal administrative region, Dist = county administrative region, Town = township administrative region, Comm = village administrative region, Vill = village group or production team, Poi = point of interest, SPoi = sub-point of interest, Road = road, RoadNo = building number of road, HouseNo = building number, UnitNo = building unit number, FloorNo = floor number, RoomNo = room number.

[0059] It has the following steps:

[0060] Execute S1 and train multiple address unit annotation models;

[0061] Based on the training set ADS train The labeled address text in the dataset is used to train address units to label models Model1, Model2, Model3, Model4, Model5, and Model6 using CRF, Bi-GRU, Bi-GRU+CRF, Bert, Bert+CRF, and Bert+Bi-GRU+CRF, respectively.

[0062] Execute S2 and process data using a multiple address unit annotation model;

[0063] Label the model using trained address units. i Processing test set ADS test The text in the ADS, 1≤i≤6, is... test AddItem, Model, any address unit of the text Str to be processed i The processing result is represented as (Tag) i Score i ), where Tag i and Score i They are using Model i The address type and score obtained after reasoning.

[0064] Execute S3 and fuse the processing results of multiple address unit annotation models;

[0065] Step S3 specifically includes:

[0066] ADS test set combines absolute voting and maximum score methods. test The six processing results (Tag) of any address unit AddItem in the middle i Score i ), 1≤i≤6.

[0067] Calculate Tag i Count of occurrences of the corresponding address type i If its maximum value exceeds 3, then the address type corresponding to that maximum value is set to the address type Tag of the fusion result. meg Satisfy Tag i ==Tag meg Time-division score i The maximum value is set as the score of the fusion result. meg If its maximum value does not exceed 3, then the address type corresponding to the maximum score will be set to the address type Tag of the fusion result. meg Set the maximum score to the score of the fusion result. meg According to Tag meg and Score meg Determine the fusion result of the address unit AddItem (Tag) meg Score meg ).

[0068] The fusion process is demonstrated using the address units "Lucheng" and "Puxieshi Xincun" in the text "Xiao Zhang's mailing address is Room 301, 3rd Floor, Unit 2, Building 8, Puxieshi Xincun, Lucheng District, Wenzhou, Zhejiang" in the test set as an example. The fusion process for other address units in the test set is the same.

[0069] Address Unit Labeling Model i The processing results for the address units “Lucheng” and “Puxieshi Xincun” are shown in Table 1, where 1≤i≤6.

[0070] Table 1. Processing results of "Puxieshi New Village" in the address unit "Lucheng" under different models.

[0071]

[0072]

[0073] Execute S33 and read the 6 processing results (Tags) of "Lucheng". i Score i ), 1≤i≤6.

[0074] Execute S34 and calculate Tag i Count of occurrences in the 6 address types i , 1≤i≤6, the results are as follows:

[0075] Count1=2, Count2=4, Count3=2, Count4=4, Count5=4, Count6=4.

[0076] Execute S35 and calculate Count. i maximum value

[0077] Execute S36, because satisfy Therefore, proceed to step S37;

[0078] Execute S37 to calculate the address type number that appears most frequently.

[0079] Execute S38 to determine the merged address type Tag. meg =Tag2=Dist;

[0080] Execute S39 to determine the merged score. Transform into S3D steps;

[0081] Perform S3D and determine the fusion result of "Lucheng" (Dist, 0.832).

[0082] The integration process of the address unit "Puxieshi New Village" is as follows:

[0083] Execute S33 and read the 6 processing results (Tags) for "Puxieshi New Village". i Score i ), 1≤i≤6.

[0084] Execute S34 and calculate Tag i Count of occurrences in the 6 address types i , 1≤i≤6, the results are as follows:

[0085] Count1=1, Count2=1, Count3=3, Count4=1, Count5=3, Count6=3.

[0086] Execute S35 and calculate Count. i maximum value

[0087] Execute S36, because Not satisfied Therefore, proceed to step S3A;

[0088] Execute S3A and calculate the address type number with the highest score.

[0089] Execute S3B and determine the merged address type Tag. meg =Tag5=POI;

[0090] Execute S3C and determine the merged score.

[0091] Perform S3D and determine the fusion result of "Puxieshi New Village" (POI, 0.811).

[0092] Execute S4 to extract address unit annotation information from the fusion result.

[0093] This embodiment extracts address unit entities and their types from the text to be processed based on the address unit annotation results. Taking the annotation result "Xiao Zhang / (O,-1) of / (O,-1) recipient / (O,-1) address / (O,-1) is / (O,-1) Zhejiang / (Prov,0.915) Wenzhou / (City,0.879) Lucheng / (Dist,0.832) Puxieshi Xincun / (POI,0.811) Building 8 / (HouseNo,0.782) Unit 2 / (UnitNo,0.776) 3rd floor / (FloorNo,0.756) Room 301 / (RoomNo,0.851). / (O,-1)" as an example, the extracted results are as follows:

[0094] Tag(Zhejiang) = Prov, Score(Zhejiang) = 0.915, Tag(Wenzhou) = City, Score(Wenzhou) = 0.879, Tag(Lucheng) = Dist, Score(Lucheng) = 0.832, Tag(Puxieshi Xincun) = POI, Score(Puxieshi Xincun) = 0.811, Tag(Building 8) = HouseNo, Score(Building 8) = 0.782, Tag(Unit 2) = UnitNo, Score(Unit 2) = 0.776, Tag(Floor 3) = FloorNo, Score(Floor 3) = 0.756, Tag(Room 301) = RoomNo, Score(Room 301) = 0.851.

[0095] Based on the above methods, such as Figure 3 As shown, an address unit parsing device in this embodiment is obtained, including an address unit annotation model training component, an address type and score generation component, an address type and score fusion component, and an address unit annotation information extraction component.

[0096] Address unit annotation model training component, used to train multiple address unit annotation models;

[0097] The address type and score generation component is used to obtain multiple address types and scores for each address unit;

[0098] Address type and score fusion component, used to fuse multiple address types and scores for each address unit;

[0099] The address unit annotation information extraction component is used to extract the address unit entity, address type, and score from the fusion result.

[0100] The address type and score fusion component includes an absolute voting method fusion sub-component, a maximum score method fusion sub-component, a fusion method selection sub-component, and a fusion result generation sub-component.

[0101] The absolute voting method fusion sub-component is used to fuse multiple address types and scores for each address unit using a comprehensive voting method;

[0102] The maximum score fusion sub-component is used to fuse multiple address types and scores for each address unit using the maximum score method;

[0103] The fusion method selection sub-component is used to select the fusion address type and score based on the voting results.

[0104] The fusion result generation sub-component is used to generate the fusion result for each address unit based on the fused address type and score.

[0105] The specific embodiments described above are merely specific examples of the present invention. The patent protection scope of the present invention includes, but is not limited to, the specific embodiments described above. Any appropriate changes or substitutions made by a person skilled in the art that conform to the claims of the address unit resolution method and apparatus of the present invention should fall within the patent protection scope of the present invention.

[0106] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method of address unit resolution, comprising: Having the following steps: S1, training a plurality of address unit labeling models; Using the training set ADS train Training M address unit labeling model Model i , 1≤i≤M; S2, processing data using the plurality of address unit labeling models; using the trained address unit labeling model Model i processing the text in the test set ADS test 1≤i≤M; For ADS test , any address unit AddItem of the text Str to be processed, Model i , the processing result is represented as (Tag i , Score i ), wherein Tag i and Score i are the address type and score obtained after using Model i to infer, and for other entities other than the address unit, Tag i = O, Score i = -1; S3, fusing processing results of the plurality of address unit labeling models; M fusion test set ADS test M processing results (Tag i ,Score i ) of any address unit AddItem, 1≤i≤M; Tag meg , Score meg ) represents the fusion result, |x| represents the integer operation on x, Count i represents the number of occurrences of the address type Tag i , k represents the address type number with the most occurrences, and t represents the address type number with the maximum score; Specifically comprising: S31, read ADS test the first text Str in the middle S32, reading the first address unit AddItem in Str; S33, read M processing results (Tag i , Score i ) of AddItem, 1≤i≤M; S34, calculate Tag i Count of occurrences in M address types i , 1≤i≤M; S35, calculate Count i maximum value of the count S36, if then go to step S37, else go to step S3A; S37、calculate the address type number with the most occurrences S38, determine the address type Tag after fusion meg = Tag k ; S39、According to the principle of the number of occurrences to determine the score after fusion Go to step S3D; S3A: Calculate the address type number with the highest score S3B: Determine the address type Tag after fusion meg = Tag t ; S3C: determining the fused score according to the maximum score principle S3D: determine the fusion result (Tag meg , Score meg ) of AddItem; S3E: If there are still unprocessed address units AddItem in Str, go to step S3F, otherwise go to step S3G; S3F: Read the next address unit AddItem in Str, go to step S33; S3G: If there is still unprocessed text to be processed in the ADS test S3H, otherwise go to step S3I; S3H: read ADS test Next, the text Str is processed, and the process goes to step S32. S3I: Return test set ADS test The fusion result of all address units in the middle S4, extracting address unit labeling information from the fusion result.

2. The method of claim 1, wherein, In step S4, according to the fusion result, the address unit entity, address type and score are extracted from the text to be processed.

3. An address unit resolution apparatus, characterized by, The method of any one of claims 1-2, comprising an address unit labeling model training component, an address type and score generation component, an address type and score fusion component, and an address unit labeling information extraction component; The address unit labeling model training component is configured to train a plurality of address unit labeling models; The address type and score generation component is configured to obtain a plurality of address types and scores for each address unit; The address type and score fusion component is configured to fuse the plurality of address types and scores for each address unit; The address unit labeling information extraction component is configured to extract the address unit entity, address type and score from the fusion result.

4. An address unit resolution apparatus as claimed in claim 3, characterized in that The address type and score fusion component comprises an absolute voting method fusion sub-component, a maximum score method fusion sub-component, a fusion method selection sub-component and a fusion result generation sub-component; The absolute voting method fusion sub-component is configured to fuse the plurality of address types and scores for each address unit using the comprehensive voting method; The maximum score method fusion sub-component is configured to fuse the plurality of address types and scores for each address unit using the maximum score method; The fusion method selection sub-component is configured to select a method for fusing the address types and scores according to the voting result; The fusion result generation sub-component is configured to generate the fusion result for each address unit according to the fused address types and scores.

Images