A fast fine-tuning method and system for address classification based on text classification

Abstract

The present invention discloses a method and system for fast fine-tuning address classification based on text classification, the method comprising: obtaining address text to be classified; determining the regional parameters corresponding to the address text according to the address text and preset keyword recognition rules; screening corresponding regional training data from a preset training database according to the regional parameters; fine-tuning a pre-trained language model according to the regional training data to obtain a fine-tuned target classification model; the target classification model is used to classify the address text. It can be seen that the present invention can effectively improve the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reduce classification errors.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for quickly fine-tuning address classification based on text classification. Background Art

[0002] Address standardization is a meticulous process that involves the precise extraction, parsing, and verification of raw, differently formatted address data, and ultimately reconstructing it into a unified and standardized format. This process not only significantly enhances the availability and consistency of data, but also plays a vital role in the efficient operation of key areas such as logistics, urban planning, and emergency response. Specifically, the address standardization process includes several key steps. First, it requires precise processing of the input address information, which may be in different formats and contain different geographic elements. Next, through a series of precise processing steps, the raw address data is converted into outputs with a unified format that clearly reflect the various geographic elements in the address, such as provinces, cities, districts, streets, and towns / communities. Most of the existing algorithms for address classification are implemented in the form of address fences, without considering the use of text classification algorithms and fine-tuning techniques for pre-trained models to improve the accuracy of address classification. Therefore, the classification effect is poor when dealing with text address classification tasks. It can be seen that the existing technology has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for rapid fine-tuning of address classification based on text classification, which can effectively improve the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified and reduce classification errors.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a method for fast fine-tuning address classification based on text classification, the method comprising:

[0005] Get the address text to be classified;

[0006] Determine the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule;

[0007] According to the regional parameters, corresponding regional training data are screened out from a preset training database;

[0008] The pre-trained language model is fine-tuned according to the regional training data to obtain a fine-tuned target classification model; the target classification model is used to perform address classification on the address text.

[0009] As an optional implementation, in the first aspect of the present invention, determining the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule includes:

[0010] According to the first keyword rule corresponding to the standard regional planning level, the address text is identified to obtain multiple first keywords;

[0011] According to the second keyword rule corresponding to the non-standard area planning, the address text is identified to obtain multiple second keywords;

[0012] Determine, according to the first keyword, the regional level complexity and regional location range corresponding to the address text;

[0013] Determining the address detail and address missingness corresponding to the address text according to the second keyword;

[0014] The area level complexity, the area location range, the address missing degree and the address detail are determined as area parameters corresponding to the address text.

[0015] As an optional implementation, in the first aspect of the present invention, determining the area level complexity and area location range corresponding to the address text according to the first keyword includes:

[0016] Input each of the first keywords into a trained distinction level classifier to obtain a region level corresponding to each of the first keywords;

[0017] Calculate the standard deviation values ​​between the regional levels corresponding to all the first keywords to obtain the regional level complexity corresponding to the address text;

[0018] Marking a keyword position range corresponding to each of the first keywords on the preset map data; the keyword position range is calculated by a corresponding mathematical relationship between the preset keywords and the position range;

[0019] Calculate the intersection between the keyword position ranges corresponding to any two of the first keywords to obtain multiple intersection areas;

[0020] According to the multiple intersection areas, the area location range corresponding to the address text is determined.

[0021] As an optional implementation, in the first aspect of the present invention, determining the area location range corresponding to the address text according to the multiple intersection areas includes:

[0022] For each intersection area, calculate the number of all the keyword position ranges to which the intersection area belongs;

[0023] Calculate the area of ​​the intersection region;

[0024] Calculate the area distance between the intersection area and the nearest other intersection area;

[0025] Calculate the product of the quantity, the area of ​​the region and the distance between the regions to obtain the region priority;

[0026] Based on a dynamic programming algorithm, a maximum polygonal area including all the intersection areas whose area priorities are greater than a first priority threshold is calculated to obtain the area location range corresponding to the address text.

[0027] As an optional implementation, in the first aspect of the present invention, determining the address detail and address missingness corresponding to the address text according to the second keyword includes:

[0028] Input each of the second keywords into a trained classification fineness classifier to obtain a classification fineness corresponding to each of the second keywords;

[0029] Calculate the weighted sum average of the number of occurrences corresponding to the division fineness corresponding to all the second keywords to obtain the address detail corresponding to the address text; wherein the weight of each of the weighted calculations of the number of occurrences is proportional to the value of the corresponding division fineness; and the value of the division fineness is proportional to the degree of address detail that can be represented by the second keyword;

[0030] The number of standard division finenesses whose appearance times in all the division finenesses corresponding to the second keyword are lower than the number threshold among the preset multiple standard division finenesses is calculated to obtain the address missing degree corresponding to the address text.

[0031] As an optional implementation, in the first aspect of the present invention, the step of selecting corresponding regional training data from a preset training database according to the regional parameters includes:

[0032] For each candidate training data in a preset training database, obtaining a region annotation parameter corresponding to the candidate training data;

[0033] Calculating the similarity between the region annotation parameter and the region parameter to obtain the data priority of the candidate training data;

[0034] Arrange all the candidate training data from large to small according to the data priority to obtain a data sequence;

[0035] All the candidate training data of a preset number in the data sequence and whose data priority is greater than a second priority threshold are determined as regional training data.

[0036] As an optional implementation, in the first aspect of the present invention, fine-tuning the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model includes:

[0037] Dividing the regional training data into a training data set and a validation data set;

[0038] Input the training data set into the pre-trained BERT model for fine-tuning training, and update the model parameters of the BERT model with the cross entropy loss function as the target in the fine-tuning training;

[0039] The BERT model after fine-tuning training is verified and calculated based on the verification data set to calculate the prediction accuracy. When the prediction accuracy is greater than a preset accuracy threshold, the BERT model is determined as the fine-tuned target classification model.

[0040] As an optional embodiment, in the first aspect of the present invention, the BERT model is pre-trained with a Chinese data set, and macbert-large-chinese is used as its initial weight during the fine-tuning process; the BERT model includes a 24-layer Transformer structure, has 1024 hidden units, and 16 self-attention heads; the fine-tuning training is implemented based on the deep learning framework paddle, using the AdamW optimizer, the maximum text input sequence length is set to 64, 3 epochs of training are performed, a dynamic learning rate is used, and the initial value of the learning rate is set to 1.5e-5.

[0041] A second aspect of an embodiment of the present invention discloses a system for fast fine-tuning address classification based on text classification, the system comprising:

[0042] An acquisition module is used to acquire address text to be classified;

[0043] A determination module, used to determine the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule;

[0044] A screening module, used to screen out corresponding regional training data from a preset training database according to the regional parameters;

[0045] A fine-tuning module is used to fine-tune the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model; the target classification model is used to perform address classification on the address text.

[0046] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule includes:

[0047] According to the first keyword rule corresponding to the standard regional planning level, the address text is identified to obtain multiple first keywords;

[0048] According to the second keyword rule corresponding to the non-standard area planning, the address text is identified to obtain multiple second keywords;

[0049] Determine, according to the first keyword, the regional level complexity and regional location range corresponding to the address text;

[0050] Determining the address detail and address missingness corresponding to the address text according to the second keyword;

[0051] The area level complexity, the area location range, the address missing degree and the address detail are determined as area parameters corresponding to the address text.

[0052] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the regional level complexity and regional location range corresponding to the address text according to the first keyword includes:

[0053] Input each of the first keywords into a trained distinction level classifier to obtain a region level corresponding to each of the first keywords;

[0054] Calculate the standard deviation values ​​between the regional levels corresponding to all the first keywords to obtain the regional level complexity corresponding to the address text;

[0055] Marking a keyword position range corresponding to each of the first keywords on the preset map data; the keyword position range is calculated by a corresponding mathematical relationship between the preset keywords and the position range;

[0056] Calculate the intersection between the keyword position ranges corresponding to any two of the first keywords to obtain multiple intersection areas;

[0057] According to the multiple intersection areas, the area location range corresponding to the address text is determined.

[0058] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the area location range corresponding to the address text according to the multiple intersection areas includes:

[0059] For each intersection area, calculate the number of all the keyword position ranges to which the intersection area belongs;

[0060] Calculate the area of ​​the intersection region;

[0061] Calculate the area distance between the intersection area and the nearest other intersection area;

[0062] Calculate the product of the quantity, the area of ​​the region and the distance between the regions to obtain the region priority;

[0063] Based on a dynamic programming algorithm, a maximum polygonal area including all the intersection areas whose area priorities are greater than a first priority threshold is calculated to obtain the area location range corresponding to the address text.

[0064] As an optional implementation, in the second aspect of the present invention, the specific manner in which the determination module determines the address detail and address missingness corresponding to the address text according to the second keyword includes:

[0065] Input each of the second keywords into a trained classification fineness classifier to obtain a classification fineness corresponding to each of the second keywords;

[0066] Calculate the weighted sum average of the number of occurrences corresponding to the division fineness corresponding to all the second keywords to obtain the address detail corresponding to the address text; wherein the weight of each of the weighted calculations of the number of occurrences is proportional to the value of the corresponding division fineness; and the value of the division fineness is proportional to the degree of address detail that can be represented by the second keyword;

[0067] The number of standard division finenesses whose appearance times in all the division finenesses corresponding to the second keyword are lower than the number threshold among the preset multiple standard division finenesses is calculated to obtain the address missing degree corresponding to the address text.

[0068] As an optional implementation, in the second aspect of the present invention, the specific manner in which the screening module screens out the corresponding regional training data from a preset training database according to the regional parameters includes:

[0069] For each candidate training data in a preset training database, obtaining a region annotation parameter corresponding to the candidate training data;

[0070] Calculating the similarity between the region annotation parameter and the region parameter to obtain the data priority of the candidate training data;

[0071] Arrange all the candidate training data from large to small according to the data priority to obtain a data sequence;

[0072] All the candidate training data of a preset number in the data sequence and whose data priority is greater than a second priority threshold are determined as regional training data.

[0073] As an optional implementation, in the second aspect of the present invention, the fine-tuning module fine-tunes the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model in a specific manner, including:

[0074] Dividing the regional training data into a training data set and a validation data set;

[0075] Input the training data set into the pre-trained BERT model for fine-tuning training, and update the model parameters of the BERT model with the cross entropy loss function as the target in the fine-tuning training;

[0076] The BERT model after fine-tuning training is verified and calculated based on the verification data set to calculate the prediction accuracy. When the prediction accuracy is greater than a preset accuracy threshold, the BERT model is determined as the fine-tuned target classification model.

[0077] As an optional embodiment, in the second aspect of the present invention, the BERT model is pre-trained with a Chinese data set, and macbert-large-chinese is used as its initial weight during the fine-tuning process; the BERT model includes a 24-layer Transformer structure, has 1024 hidden units, and 16 self-attention heads; the fine-tuning training is implemented based on the deep learning framework paddle, using the AdamW optimizer, the maximum text input sequence length is set to 64, 3 epochs of training are performed, a dynamic learning rate is used, and the initial value of the learning rate is set to 1.5e-5.

[0078] The third aspect of the present invention discloses another fast fine-tuning system for address classification based on text classification, the system comprising:

[0079] A memory storing executable program code;

[0080] a processor coupled to the memory;

[0081] The processor calls the executable program code stored in the memory to execute part or all of the steps in the method for fast fine-tuning address classification based on text classification disclosed in the first aspect of the present invention.

[0082] The fourth aspect of the present invention discloses a computer storage medium storing computer instructions, which, when called, are used to execute part or all of the steps in the method for rapid fine-tuning of address classification based on text classification disclosed in the first aspect of the present invention.

[0083] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0084] The present invention can determine the regional parameters of the address text, and then use this to screen out the corresponding regional training data to quickly and in real time fine-tune the pre-trained language model, so that the target classification model can more accurately and specifically classify the address text, thereby effectively improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0086] Figure 1 The present invention is a flowchart of a method for quickly fine-tuning address classification based on text classification disclosed in an embodiment of the present invention.

[0087] Figure 2 It is a structural diagram of a rapid fine-tuning system for address classification based on text classification disclosed in an embodiment of the present invention.

[0088] Figure 3 It is a structural schematic diagram of another system for fast fine-tuning address classification based on text classification disclosed in an embodiment of the present invention.

[0089] Figure 4 It is a structural schematic diagram of a fast fine-tuning model disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0091] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0092] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0093] The present invention discloses a method and system for fast fine-tuning address classification based on text classification, which can determine the regional parameters of the address text, and then screen out the corresponding regional training data to quickly and in real time fine-tune the pre-trained language model, so that the target classification model can more accurately and targetedly classify the address text, thereby effectively improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors. The following are detailed descriptions.

[0094] Embodiment 1

[0095] See also Figure 1 , Figure 1 : is a flowchart of a method for fast fine-tuning address classification based on text classification disclosed in an embodiment of the present invention. Figure 1 The described method for fast fine-tuning address classification based on text classification can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the fast fine-tuning method for address classification based on text classification may include the following operations:

[0096] 101. Obtain address text to be classified.

[0097] 102. Determine the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule.

[0098] 103. According to the regional parameters, corresponding regional training data is selected from a preset training database.

[0099] 104. Fine-tune the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model.

[0100] Optionally, a target classification model is used to classify the address text.

[0101] It can be seen that the above-mentioned embodiments of the invention can determine the regional parameters of the address text, and then use this to filter out the corresponding regional training data to quickly and in real time fine-tune the pre-trained language model, so that the target classification model can more accurately and specifically classify the address text, thereby effectively improving the targetedness and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0102] As an optional embodiment, in the above step, determining the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule includes:

[0103] According to the first keyword rule corresponding to the standard regional planning level, the address text is recognized to obtain multiple first keywords;

[0104] According to the second keyword rule corresponding to the non-standard area planning, the address text is recognized to obtain multiple second keywords;

[0105] According to the first keyword, determine the regional level complexity and regional location range corresponding to the address text;

[0106] According to the second keyword, determining the address detail and address missingness corresponding to the address text;

[0107] The regional level complexity, regional location range, address missingness and address detail are determined as regional parameters corresponding to the address text.

[0108] It can be seen that through the above optional embodiments, the corresponding keywords can be identified through the keyword rules corresponding to the standard regional planning level and the non-standard regional planning, and the regional level complexity, regional location range, address missingness and address detail can be calculated to determine the regional parameters corresponding to the address text, so as to facilitate the subsequent screening of accurate regional training data for fast and real-time fine-tuning of the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0109] As an optional embodiment, in the above step, determining the area level complexity and area location range corresponding to the address text according to the first keyword includes:

[0110] Input each first keyword into the trained distinction level classifier to obtain the region level corresponding to each first keyword;

[0111] Calculate the standard deviation between the regional levels corresponding to all first keywords to obtain the regional level complexity corresponding to the address text;

[0112] Marking the keyword position range corresponding to each first keyword on the preset map data; optionally, the keyword position range is calculated by a corresponding mathematical relationship between the preset keywords and the position range;

[0113] Calculate the intersection between the keyword position ranges corresponding to any two first keywords to obtain multiple intersection areas;

[0114] According to the multiple intersection areas, the area location range corresponding to the address text is determined.

[0115] It can be seen that through the above-mentioned optional embodiments, the keywords can be classified by the distinction level classifier to obtain the distinction level, and the level complexity can be determined by standard deviation calculation, and the regional location range can be determined by the intersection calculation of the keyword location range, so as to facilitate the subsequent screening of accurate regional training data for fast and real-time fine-tuning of the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0116] As an optional embodiment, in the above step, determining the area location range corresponding to the address text according to the multiple intersection areas includes:

[0117] For each intersection region, calculate the number of all keyword position ranges to which the intersection region belongs;

[0118] Calculate the area of ​​the intersection region;

[0119] Calculate the area distance between the intersection area and the nearest other intersection area;

[0120] Calculate the product between the quantity, the area of ​​the region and the distance between the regions to obtain the region priority;

[0121] Based on the dynamic programming algorithm, the maximum polygonal area including the intersection area of ​​all the area priorities greater than the first priority threshold is calculated to obtain the area location range corresponding to the address text.

[0122] It can be seen that through the above optional embodiments, the regional priority can be obtained by calculating multiple parameters of the intersection area, and based on this, the area is screened and the polygon calculation of the range is performed to determine the precise area location range, which is convenient for subsequent screening of accurate regional training data to quickly and in real time fine-tune the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0123] As an optional embodiment, in the above step, determining the address detail level and the address missing level corresponding to the address text according to the second keyword includes:

[0124] Input each second keyword into the trained segmentation fineness classifier to obtain the segmentation fineness corresponding to each second keyword;

[0125] Calculate the weighted sum average of the number of occurrences corresponding to the division fineness corresponding to all the second keywords to obtain the address detail corresponding to the address text; optionally, the weight of each number of occurrences in the weighted calculation is proportional to the value of the corresponding division fineness; the value of the division fineness is proportional to the degree of address detail that can be represented by the second keyword;

[0126] The number of standard division finenesses whose appearance times in all division finenesses corresponding to the second keyword are lower than the number threshold among the preset multiple standard division finenesses is calculated to obtain the address missing degree corresponding to the address text.

[0127] It can be seen that through the above optional embodiments, the keywords can be classified by the division fineness classifier to obtain the division fineness, and the address detail and address missingness can be determined by calculating the number of occurrences, which is convenient for the subsequent screening of accurate regional training data to quickly and in real time fine-tune the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0128] As an optional embodiment, in the above step, selecting corresponding regional training data from a preset training database according to regional parameters includes:

[0129] For each candidate training data in a preset training database, obtaining a region annotation parameter corresponding to the candidate training data;

[0130] Calculate the similarity between the region annotation parameters and the region parameters to obtain the data priority of the candidate training data;

[0131] Arrange all candidate training data from large to small according to data priority to obtain a data sequence;

[0132] All candidate training data of a preset number in the data sequence and whose data priority is greater than a second priority threshold are determined as regional training data.

[0133] It can be seen that through the above optional embodiments, by calculating the similarity between regional parameters and the sorting threshold screening step based thereon, more targeted regional training data can be screened out from the preset training database to quickly and in real time fine-tune the pre-trained language model, thereby assisting in improving the targetedness and accuracy of address classification according to the regional characteristics of the text to be classified and reducing classification errors.

[0134] As an optional embodiment, in the above step, fine-tuning the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model includes:

[0135] Divide the regional training data into a training data set and a validation data set;

[0136] Input the training data set into the pre-trained BERT model for fine-tuning training, and update the model parameters of the BERT model with the cross entropy loss function as the target during fine-tuning training;

[0137] The fine-tuned BERT model is verified based on the verification data set to calculate the prediction accuracy. When the prediction accuracy is greater than the preset accuracy threshold, the BERT model is determined as the fine-tuned target classification model.

[0138] It can be seen that through the above optional embodiments, the regional training data can be divided into a training data set and a verification data set to train and verify the pre-trained BERT model, so as to quickly fine-tune the target classification model, thereby improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified and reducing classification errors.

[0139] As an optional embodiment, in the above steps, the BERT model is pre-trained with a Chinese data set, and macbert-large-chinese is used as its initial weight during the fine-tuning process; the BERT model contains a 24-layer Transformer structure, has 1024 hidden units, and 16 self-attention heads; the fine-tuning training is implemented based on the deep learning framework paddle, using the AdamW optimizer, the maximum text input sequence length is set to 64, and 3 epochs of training are performed, using a dynamic learning rate, and the initial value of the learning rate is set to 1.5e-5.

[0140] Specifically, the BERT model is a classic deep learning model based on the Transformer architecture. It has learned rich language knowledge and context understanding capabilities through pre-training. The overall structure of the BERT model can be summarized as input representation (Input Embeddings) to multi-head attention (Multi-Head Attention), then to residual connection and layer normalization (LayerNorm), and finally to the feed forward network (Feed Forward), and ends with residual connection and layer normalization.

[0141] Specifically, the pre-trained models have successfully acquired rich language representation knowledge and excellent context understanding capabilities through in-depth training on large-scale text datasets. When applied to specific address community classification tasks, these models can effectively use their prior knowledge to keenly identify subtle differences in address texts, thereby greatly improving classification accuracy and efficiency.

[0142] In a specific implementation plan, the Ernie model was selected as the basic pre-training model. Its overall architecture is similar to BERT, and it is also a pre-training language model based on the Transformer architecture. Compared with BERT, the Ernie model integrates external knowledge such as entities, phrases, and language structures during the pre-training process, and implements entity-level masking including three levels: words, entities, and phrases, so as to better understand the semantic information in the language. By training the model by masking words, entities, etc., Ernie is able to learn richer language representations. In addition, in addition to the MLM and NSP tasks used by BERT, Ernie introduces additional pre-training tasks to enhance the model's understanding of language structure and semantics. These tasks help the Ernie model better understand the relationship between vocabulary and the deep meaning of language.

[0143] In this implementation scheme, after analyzing the regional parameters of the address text to be classified and matching it to the corresponding regional training data, a pre-trained language model + classification layer is used to construct a fine-tuning model. For details, see Figure 4 , During fine-tuning, for the input text in the regional training data, , fill the text with [CLS] symbols before and [SEP] symbols after the text and align them to the input length of the pre-trained language model (such as BERT, Ernie) with [PAD]. The model embeds all word units W={〖w_CLS,w〗_1,w_2,⋯} into vectors and inputs them into the pre-trained language model (such as BERT, Ernie), and obtains the output sequence O={O_CLS,O_1,O_2,⋯}. Among them, O_CLS is the hidden state vector of the [CLS] symbol, which represents the complete address semantics. O_CLS is sent to the classification layer for optimization to obtain the address label corresponding to the text.

[0144] In this embodiment, the fine-tuning model can be described as follows: }, the output is .

[0145] Accordingly, the predicted probability distribution of the geographic area label of this address is:

[0146] ;

[0147] During the training process, considering the multi-classification characteristics of geographic area labels, the cross entropy loss function is used as the optimization objective: ;

[0148] in Represents the true geographic region label of the sample, where for each sample, the correct category position is 1 and other positions are 0. is the output of the model prediction, which represents the predicted probability distribution of the model predicted geographic area. Each element Indicates The samples belong to The predicted probability of the class, is the total number of categories, that is, the number of different geographic regions that the model needs to distinguish, Is the real label Does the sample belong to The indicator of the class, if the sample belongs to this class, it is 1, otherwise it is 0. The model predicts The samples belong to The probability of the class.

[0149] In the specific experimental verification of this implementation scheme, it was found that the above-mentioned address classification algorithm model obtained by adopting the fast fine-tuning technology in the present invention can achieve a level that exceeds the classical method in a window test set with relatively good data quality, compared with the classical geocoding API.

[0150] It can be seen that through the above optional embodiments, the training details and model details of the BERT model are clarified, so as to quickly fine-tune the target classification model, thereby improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified and reducing classification errors.

[0151] Embodiment 2

[0152] See also Figure 2 , Figure 2 : is a schematic diagram of a structure of a fast fine-tuning system for address classification based on text classification disclosed in an embodiment of the present invention. Figure 2The described fast fine-tuning system for address classification based on text classification can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the fast fine-tuning system for address classification based on text classification may include:

[0153] The acquisition module 201 is used to acquire the address text to be classified.

[0154] The determination module 202 is used to determine the area parameter corresponding to the address text according to the address text and the preset keyword recognition rule.

[0155] The screening module 203 is used to screen out corresponding regional training data from a preset training database according to regional parameters.

[0156] The fine-tuning module 204 is used to fine-tune the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model.

[0157] Optionally, a target classification model is used to classify the address text.

[0158] It can be seen that the above-mentioned embodiments of the invention can determine the regional parameters of the address text, and then use this to filter out the corresponding regional training data to quickly and in real time fine-tune the pre-trained language model, so that the target classification model can more accurately and specifically classify the address text, thereby effectively improving the targetedness and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0159] As an optional embodiment, the specific manner in which the determination module determines the area parameter corresponding to the address text according to the address text and the preset keyword recognition rule includes:

[0160] According to the first keyword rule corresponding to the standard regional planning level, the address text is recognized to obtain multiple first keywords;

[0161] According to the second keyword rule corresponding to the non-standard area planning, the address text is recognized to obtain multiple second keywords;

[0162] According to the first keyword, determine the regional level complexity and regional location range corresponding to the address text;

[0163] According to the second keyword, determining the address detail and address missingness corresponding to the address text;

[0164] The regional level complexity, regional location range, address missingness and address detail are determined as regional parameters corresponding to the address text.

[0165] It can be seen that through the above optional embodiments, the corresponding keywords can be identified through the keyword rules corresponding to the standard regional planning level and the non-standard regional planning, and the regional level complexity, regional location range, address missingness and address detail can be calculated to determine the regional parameters corresponding to the address text, so as to facilitate the subsequent screening of accurate regional training data for fast and real-time fine-tuning of the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0166] As an optional embodiment, the specific manner in which the determination module determines the regional level complexity and the regional location range corresponding to the address text according to the first keyword includes:

[0167] Input each first keyword into the trained distinction level classifier to obtain the region level corresponding to each first keyword;

[0168] Calculate the standard deviation between the regional levels corresponding to all first keywords to obtain the regional level complexity corresponding to the address text;

[0169] Marking the keyword position range corresponding to each first keyword on the preset map data; optionally, the keyword position range is calculated by a corresponding mathematical relationship between the preset keywords and the position range;

[0170] Calculate the intersection between the keyword position ranges corresponding to any two first keywords to obtain multiple intersection areas;

[0171] According to the multiple intersection areas, the area location range corresponding to the address text is determined.

[0172] It can be seen that through the above-mentioned optional embodiments, the keywords can be classified by the distinction level classifier to obtain the distinction level, and the level complexity can be determined by standard deviation calculation, and the regional location range can be determined by the intersection calculation of the keyword location range, so as to facilitate the subsequent screening of accurate regional training data for fast and real-time fine-tuning of the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0173] As an optional embodiment, the specific manner in which the determination module determines the area location range corresponding to the address text according to the multiple intersection areas includes:

[0174] For each intersection region, calculate the number of all keyword position ranges to which the intersection region belongs;

[0175] Calculate the area of ​​the intersection region;

[0176] Calculate the area distance between the intersection area and the nearest other intersection area;

[0177] Calculate the product between the quantity, the area of ​​the region and the distance between the regions to obtain the region priority;

[0178] Based on the dynamic programming algorithm, the maximum polygonal area including the intersection area of ​​all the area priorities greater than the first priority threshold is calculated to obtain the area location range corresponding to the address text.

[0179] It can be seen that through the above optional embodiments, the regional priority can be obtained by calculating multiple parameters of the intersection area, and based on this, the area is screened and the polygon calculation of the range is performed to determine the precise area location range, which is convenient for subsequent screening of accurate regional training data to quickly and in real time fine-tune the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0180] As an optional embodiment, the specific manner in which the determination module determines the address detail and address missingness corresponding to the address text according to the second keyword includes:

[0181] Input each second keyword into the trained segmentation fineness classifier to obtain the segmentation fineness corresponding to each second keyword;

[0182] Calculate the weighted sum average of the number of occurrences corresponding to the division fineness corresponding to all the second keywords to obtain the address detail corresponding to the address text; optionally, the weight of each number of occurrences in the weighted calculation is proportional to the value of the corresponding division fineness; the value of the division fineness is proportional to the degree of address detail that can be represented by the second keyword;

[0183] The number of standard division finenesses whose appearance times in all division finenesses corresponding to the second keyword are lower than the number threshold among the preset multiple standard division finenesses is calculated to obtain the address missing degree corresponding to the address text.

[0184] It can be seen that through the above optional embodiments, the keywords can be classified by the division fineness classifier to obtain the division fineness, and the address detail and address missingness can be determined by calculating the number of occurrences, which is convenient for the subsequent screening of accurate regional training data to quickly and in real time fine-tune the pre-trained language model, thereby assisting in improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified, and reducing classification errors.

[0185] As an optional embodiment, the specific manner in which the screening module screens out the corresponding regional training data from a preset training database according to the regional parameters includes:

[0186] For each candidate training data in a preset training database, obtaining a region annotation parameter corresponding to the candidate training data;

[0187] Calculate the similarity between the region annotation parameters and the region parameters to obtain the data priority of the candidate training data;

[0188] Arrange all candidate training data from large to small according to data priority to obtain a data sequence;

[0189] All candidate training data of a preset number in the data sequence and whose data priority is greater than a second priority threshold are determined as regional training data.

[0190] It can be seen that through the above optional embodiments, by calculating the similarity between regional parameters and the sorting threshold screening step based thereon, more targeted regional training data can be screened out from the preset training database to quickly and in real time fine-tune the pre-trained language model, thereby assisting in improving the targetedness and accuracy of address classification according to the regional characteristics of the text to be classified and reducing classification errors.

[0191] As an optional embodiment, the fine-tuning module fine-tunes the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model in a specific manner, including:

[0192] Divide the regional training data into a training data set and a validation data set;

[0193] Input the training data set into the pre-trained BERT model for fine-tuning training, and update the model parameters of the BERT model with the cross entropy loss function as the target during fine-tuning training;

[0194] The fine-tuned BERT model is verified based on the verification data set to calculate the prediction accuracy. When the prediction accuracy is greater than the preset accuracy threshold, the BERT model is determined as the fine-tuned target classification model.

[0195] It can be seen that through the above optional embodiments, the regional training data can be divided into a training data set and a verification data set to train and verify the pre-trained BERT model, so as to quickly fine-tune the target classification model, thereby improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified and reducing classification errors.

[0196] As an optional embodiment, the BERT model is pre-trained with a Chinese data set, and macbert-large-chinese is used as its initial weight during the fine-tuning process; the BERT model contains a 24-layer Transformer structure, has 1024 hidden units, and 16 self-attention heads; the fine-tuning training is implemented based on the deep learning framework paddle, using the AdamW optimizer, the maximum text input sequence length is set to 64, and 3 epochs of training are performed, using a dynamic learning rate, and the initial value of the learning rate is set to 1.5e-5.

[0197] It can be seen that through the above optional embodiments, the training details and model details of the BERT model are clarified, so as to quickly fine-tune the target classification model, thereby improving the pertinence and accuracy of address classification according to the regional characteristics of the text to be classified and reducing classification errors.

[0198] Embodiment 3

[0199] See also Figure 3 , Figure 3 The invention discloses another fast fine-tuning system for address classification based on text classification. Figure 3 The described fast fine-tuning system for address classification based on text classification is applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the fast fine-tuning system for address classification based on text classification may include:

[0200] A memory 301 storing executable program codes;

[0201] a processor 302 coupled to the memory 301;

[0202] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the method for fast fine-tuning address classification based on text classification described in the first embodiment.

[0203] Embodiment 4

[0204] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the method for fast fine-tuning address classification based on text classification described in the first embodiment.

[0205] Embodiment 5

[0206] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the method for rapid fine-tuning of address classification based on text classification described in the first embodiment.

[0207] The above describes specific embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0208] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0209] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0210] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0211] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0214] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0215] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0216] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0217] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0218] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0219] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0220] Finally, it should be noted that the method and system for rapid fine-tuning of address classification based on text classification disclosed in the embodiment of the present invention discloses only the preferred embodiments of the present invention, which are only used to illustrate the technical scheme of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical schemes described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical schemes from the spirit and scope of the technical schemes of the various embodiments of the present invention.

Claims

1. A fast fine-tuning method for address classification based on text classification, characterized in that: The method comprises: Get the address text to be classified; Determining the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule includes: According to the first keyword rule corresponding to the standard regional planning level, the address text is identified to obtain multiple first keywords; According to the second keyword rule corresponding to the non-standard area planning, the address text is identified to obtain multiple second keywords; Input each of the first keywords into a trained distinction level classifier to obtain a region level corresponding to each of the first keywords; Calculate the standard deviation values ​​between the regional levels corresponding to all the first keywords to obtain the regional level complexity corresponding to the address text; Marking a keyword position range corresponding to each of the first keywords on the preset map data; the keyword position range is calculated by a corresponding mathematical relationship between the preset keywords and the position range; Calculate the intersection between the keyword position ranges corresponding to any two of the first keywords to obtain multiple intersection areas; For each intersection area, calculate the number of all the keyword position ranges to which the intersection area belongs; Calculate the area of ​​the intersection region; Calculate the area distance between the intersection area and the nearest other intersection area; Calculate the product of the quantity, the area of ​​the region and the distance between the regions to obtain the region priority; Based on a dynamic programming algorithm, a maximum polygonal area including all the intersection areas whose area priorities are greater than a first priority threshold is calculated to obtain the area location range corresponding to the address text; Input each of the second keywords into a trained classification fineness classifier to obtain a classification fineness corresponding to each of the second keywords; Calculate the weighted sum average of the number of occurrences corresponding to the division fineness corresponding to all the second keywords to obtain the address detail corresponding to the address text; wherein the weight of each of the weighted calculations of the number of occurrences is proportional to the value of the corresponding division fineness; and the value of the division fineness is proportional to the degree of address detail that can be represented by the second keyword; Calculate the number of standard division finenesses whose appearance times in all the division finenesses corresponding to the second keyword are lower than the number threshold among a plurality of preset standard division finenesses, and obtain the address missing degree corresponding to the address text; Determine the area level complexity, the area location range, the address missing degree and the address detail as area parameters corresponding to the address text; According to the regional parameters, corresponding regional training data are screened out from a preset training database; The pre-trained language model is fine-tuned according to the regional training data to obtain a fine-tuned target classification model; the target classification model is used to perform address classification on the address text.

2. The method for fast fine-tuning address classification based on text classification according to claim 1 is characterized in that: The step of selecting corresponding regional training data from a preset training database according to the regional parameters includes: For each candidate training data in a preset training database, obtaining a region annotation parameter corresponding to the candidate training data; Calculating the similarity between the region annotation parameter and the region parameter to obtain the data priority of the candidate training data; Arrange all the candidate training data from large to small according to the data priority to obtain a data sequence; All the candidate training data of a preset number in the data sequence and whose data priority is greater than a second priority threshold are determined as regional training data.

3. The method for fast fine-tuning address classification based on text classification according to claim 1 is characterized in that: The fine-tuning of the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model includes: Dividing the regional training data into a training data set and a validation data set; Input the training data set into the pre-trained BERT model for fine-tuning training, and update the model parameters of the BERT model with the cross entropy loss function as the target in the fine-tuning training; The BERT model after fine-tuning training is verified and calculated based on the verification data set to calculate the prediction accuracy. When the prediction accuracy is greater than a preset accuracy threshold, the BERT model is determined as the fine-tuned target classification model.

4. The method for fast fine-tuning address classification based on text classification according to claim 3 is characterized in that: The BERT model is pre-trained with a Chinese data set, and macbert-large-chinese is used as its initial weight during the fine-tuning process; the BERT model contains a 24-layer Transformer structure, has 1024 hidden units, and 16 self-attention heads; the fine-tuning training is implemented based on the deep learning framework paddle, using the AdamW optimizer, the maximum text input sequence length is set to 64, and 3 epochs of training are performed, using a dynamic learning rate, and the initial value of the learning rate is set to 1.5e-5.

5. A fast fine-tuning system for address classification based on text classification, characterized in that: The system comprises: An acquisition module is used to acquire address text to be classified; A determination module, used to determine the area parameter corresponding to the address text according to the address text and a preset keyword recognition rule, including: According to the first keyword rule corresponding to the standard regional planning level, the address text is identified to obtain multiple first keywords; According to the second keyword rule corresponding to the non-standard area planning, the address text is identified to obtain multiple second keywords; Input each of the first keywords into a trained distinction level classifier to obtain a region level corresponding to each of the first keywords; Calculate the standard deviation values ​​between the regional levels corresponding to all the first keywords to obtain the regional level complexity corresponding to the address text; Marking a keyword position range corresponding to each of the first keywords on the preset map data; the keyword position range is calculated by a corresponding mathematical relationship between the preset keywords and the position range; Calculate the intersection between the keyword position ranges corresponding to any two of the first keywords to obtain multiple intersection areas; For each intersection area, calculate the number of all the keyword position ranges to which the intersection area belongs; Calculate the area of ​​the intersection region; Calculate the area distance between the intersection area and the nearest other intersection area; Calculate the product of the quantity, the area of ​​the region and the distance between the regions to obtain the region priority; Based on a dynamic programming algorithm, a maximum polygonal area including all the intersection areas whose area priorities are greater than a first priority threshold is calculated to obtain the area location range corresponding to the address text; Input each of the second keywords into a trained classification fineness classifier to obtain a classification fineness corresponding to each of the second keywords; Calculate the weighted sum average of the number of occurrences corresponding to the division fineness corresponding to all the second keywords to obtain the address detail corresponding to the address text; wherein the weight of each of the weighted calculations of the number of occurrences is proportional to the value of the corresponding division fineness; and the value of the division fineness is proportional to the degree of address detail that can be represented by the second keyword; Calculate the number of standard division finenesses whose appearance times in all the division finenesses corresponding to the second keyword are lower than the number threshold among a plurality of preset standard division finenesses, and obtain the address missing degree corresponding to the address text; Determine the area level complexity, the area location range, the address missing degree and the address detail as area parameters corresponding to the address text; A screening module, used to screen out corresponding regional training data from a preset training database according to the regional parameters; A fine-tuning module is used to fine-tune the pre-trained language model according to the regional training data to obtain a fine-tuned target classification model; the target classification model is used to perform address classification on the address text.

6. A fast fine-tuning system for address classification based on text classification, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the fast fine-tuning method for address classification based on text classification as described in any one of claims 1-4.

Images