A method and system for address text category prediction

Abstract

A method for address text category prediction includes: acquiring the address text to be predicted; semantically encoding the address text to obtain a first semantic encoding vector; processing the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector; decoding the second semantic encoding matrix output by the encoder multiple times using a second preset rule to obtain multiple decoded address tags; and combining the multiple decoded address tags according to the address tag system to obtain the final address category prediction result. This invention implicitly introduces the type and boundary information of address elements by adding an attention mechanism to the encoder, and models the local and global relationships between hierarchical tags in the decoder, thereby improving the accuracy of the address category prediction model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for predicting address text categories. Background Technology

[0002] Address text category prediction uses machine learning models to predict a multi-level category for a given address text. Existing methods transform this into a text multi-classification problem, employing a single classifier to predict the hierarchical category to which the address text belongs. Alternatively, they transform it into a hierarchical multi-label classification problem, setting an independent classifier for each level and merging the outputs of multiple classifiers to obtain the address text's category prediction result. These methods neglect the hierarchical relationships between labels, which often play a crucial role in address text category prediction. Furthermore, elements in address text typically have fixed boundaries and types; this information is also important for obtaining effective semantic encoding representations to improve prediction accuracy. Summary of the Invention

[0003] In view of the above problems, the present invention is proposed to provide a method and system for address text category prediction that overcomes or at least partially solves the above problems.

[0004] To address the aforementioned technical problems, the embodiments of this application disclose the following technical solutions: A method for predicting address text categories includes: S100. Obtain the address text to be predicted for address category; S200. Semantically encode the address text to obtain a first semantic encoding vector, and process the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector; S300. The second semantic encoding matrix output by the encoder is decoded multiple times using the second preset rule to obtain the address tag after multiple decodings; S400. Combine the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result.

[0005] Furthermore, in S200, a pre-trained address category prediction model encoder is used to semantically encode the address text to obtain the first semantic encoding vector.

[0006] Furthermore, in S200, the input address text is split into word sequences, the word sequences are vectorized using an embedding layer, and then semantically encoded using the Transformer Encoder module to obtain the first semantic encoding vector of the address text.

[0007] Furthermore, in S200, the first semantic encoding vector is processed using a first preset rule to obtain a second semantic encoding vector. The first preset rule includes: implicitly introducing the type and boundary information of address elements into the first semantic encoding vector using an attention mechanism to obtain the second semantic encoding vector.

[0008] Furthermore, the specific implementation method of the first preset rule is as follows: an attention mechanism is introduced to calculate the correlation between the first semantic encoding matrix and the address type matrix to obtain the weight score matrix, and then the weight score matrix is ​​multiplied by the address type matrix to obtain the second semantic encoding matrix as the final output of the encoding layer.

[0009] Furthermore, in S300, the second semantic encoding matrix output by the encoder is decoded multiple times using a second preset rule to obtain address tags after multiple decodings. The second preset rule includes: obtaining the address tag output by the i-th decoding through the input layer of the pre-trained address category prediction model decoder and vectorizing it using an embedding layer, and superimposing the global hierarchical features of the address tag system. Then, the address tag is decoded by combining the second semantic encoding vector of the address text to obtain the (i+1)-th decoding latent vector. Then, the (i+1)-th decoding latent vector is predicted using a classification layer to obtain the (i+1)-th address tag prediction result. This process is repeated multiple times until the decoding ends when the condition for ending the decoding is met.

[0010] Furthermore, the pre-trained address category prediction model decoder is used to predict the hierarchical category of the address text. It decodes the address tags of each level in descending order of category level, and merges the address tags of each level to obtain the final address category prediction result.

[0011] Furthermore, the decoding termination condition can be set to a fixed number of decoding attempts or the current decoding result as the end marker. <eos>.

[0012] Furthermore, in S400, the address label system is a tree-like hierarchical structure, and a graph-based representation method is used to obtain the global hierarchical features of the address label system.

[0013] This invention also discloses a system for address text category prediction, comprising: an address text acquisition unit, a second semantic encoding vector acquisition unit, an address tag acquisition unit, and an address text category prediction result unit; wherein: The address text acquisition unit is used to acquire the address text to be predicted for address category. The second semantic encoding vector acquisition unit is used to perform semantic encoding on the address text to obtain the first semantic encoding vector, and to process the first semantic encoding vector using a first preset rule to obtain the second semantic encoding vector. The address tag acquisition unit is used to decode the second semantic encoding matrix output by the encoder multiple times using a second preset rule to obtain the address tags after multiple decodings. The address text category prediction result unit is used to combine the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result.

[0014] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following: This invention discloses a method for address text category prediction, comprising: acquiring address text to be predicted; semantically encoding the address text to obtain a first semantic encoding vector; processing the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector; decoding the second semantic encoding matrix output by the encoder multiple times using a second preset rule to obtain multiple decoded address tags; and combining the multiple decoded address tags according to the address tag system to obtain the final address category prediction result. This invention implicitly introduces the type and boundary information of address elements by adding an attention mechanism to the encoder, and models the local and global relationships between hierarchical tags in the decoder, thereby improving the accuracy of the address category prediction model.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for predicting address text categories in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the address tag system in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the encoder structure of the address category prediction model in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the address category prediction model decoder structure in Embodiment 1 of the present invention. Detailed Implementation

[0017] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0018] To address the problems existing in the prior art, embodiments of the present invention provide a method and system for predicting address text categories. Example 1

[0019] This publicly available method for real-time address text category prediction improves upon existing methods for address category prediction tasks. Address categories refer to the combinations of tags at various levels within the address tagging system. (See reference...) Figure 2 This is a diagram illustrating the address labeling system. The first level contains 23 labels, the second level contains 267 labels, and the third level contains 906 labels. For the address text "Hongshan Youkeyuan Road Fenghuo Innovation Valley", the corresponding address category is "Commercial Residential - Industrial Park - Industrial Park", where the first-level label is "Commercial Residential", the second-level label is "Industrial Park", and the third-level label is "Industrial Park".

[0020] Existing address text category prediction methods transform it into a text multi-classification problem, using a single classifier to predict the hierarchical category to which the address text belongs. Alternatively, they transform address text category prediction into a hierarchical multi-label classification problem, setting an independent classifier for each level and merging the outputs of multiple classifiers to obtain the address text category prediction result. These methods ignore the hierarchical relationships between labels, while these potential associations between hierarchical category labels often play a crucial role in address text category prediction. Furthermore, elements in address text typically have fixed boundaries and types; this information is also important for obtaining effective semantic encoding representations to improve prediction accuracy.

[0021] To address the shortcomings of existing methods, we propose an address text category prediction method. This method incorporates an attention mechanism in the encoder to implicitly introduce the type and boundary information of address elements, and models the local and global relationships between hierarchical labels in the decoder. These approaches improve the accuracy of the address category prediction model.

[0022] Specifically, in this embodiment, a method for predicting address text categories is disclosed, such as... Figure 1 ,include: S100. Obtain the address text to be predicted for address category; specifically, in this embodiment of the application, the obtained address text is "Hongshan Post and Telecommunications Research Institute Road Fenghuo Innovation Valley", such as Figure 3 The input diagram is shown in the encoder structure diagram of the address category prediction model.

[0023] S200. Semantically encode the address text to obtain a first semantic encoding vector, and process the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector; Specifically, in S200 of this embodiment, a pre-trained address category prediction model encoder is used to semantically encode the address text to obtain a first semantic encoding vector. The input address text is split into word sequences, an embedding layer is used to vectorize the word sequences, and then the Transformer Encoder module is used to perform semantic encoding to obtain the first semantic encoding vector of the address text.

[0024] In step S200 of this embodiment, the first semantic encoding vector is processed using a first preset rule to obtain a second semantic encoding vector. The first preset rule includes: implicitly introducing the type and boundary information of address elements into the first semantic encoding vector using an attention mechanism to obtain the second semantic encoding vector. Specifically, the first preset rule is implemented as follows: an attention mechanism is introduced to calculate the correlation between the first semantic encoding matrix and the address type matrix to obtain a weight score matrix; then, the weight score matrix is ​​multiplied by the address type matrix to obtain the second semantic encoding matrix, which serves as the final output of the encoding layer. For details, please refer to [reference needed]. Figure 3 Schematic diagram of the encoder structure of the address category prediction model.

[0025] Specifically, the address type matrix implicitly contains the type information of the address, which is obtained through training by an optimization algorithm as a parameter of the model. In order to enable the semantic encoding vector to consider the implicit address information, the address type matrix is interacted with the first semantic encoding vector to obtain a second semantic encoding vector that integrates the address type and boundary information. Essentially, it is obtained by weighted summation of each implicit address type vector. Assume that the first semantic encoding matrix is S*H, the size of the address type matrix is T*H, and each row of the address type matrix can represent an implicit address type. For the sake of convenience, assume S = 1, T = 3, and H = 3. The first row T0 of the address type matrix represents the information of the administrative region, the second row T1 represents the information of the road name, and the third row T3 represents the information of the point of interest. For the first semantic encoding vector of the character "Hong" in the address text "Hongshan Youkeyuan Road, Fenghuo Innovation Valley", assume it is S0 = [0.003, 0.135, 0.178]. The size of the address type matrix is 3*3, and the weight score matrix calculated with the address type matrix is [0.87, 0.10, 0.03]. Its meaning can be interpreted as the probability that the character "Hong" belongs to the first address type is 0.87, the probability of the second address type is 0.10, and the probability of the third address type is 0.03. Therefore, the implicit address type result of the character "Hong" is T0. Similarly, the implicit address type result of the address text "Hongshan Youkeyuan Road, Fenghuo Innovation Valley" is "T0, T0, T1, T1, T1, T1, T2, T2, T2, T2, T2". The implicit address type is equivalent to directly providing the address type information to the model, and the difference between the implicit address types is equivalent to considering the boundaries of the address elements.

[0026] S300. Use the second preset rule to decode the second semantic encoding matrix output by the encoder multiple times to obtain the address labels after multiple decodings; In S300 of this embodiment, use the second preset rule to decode the second semantic encoding matrix output by the encoder multiple times to obtain the address labels after multiple decodings. The second preset rule includes: obtaining the address label of the i-th decoding output through the input layer of the decoder of the pre-trained address category prediction model and performing vectorization representation using the embedding layer, and superimposing the global hierarchical features of the address label system. Then, combine the second semantic encoding vector of the address text to decode the (i + 1)-th decoding hidden vector, and then use the classification layer to predict the (i + 1)-th decoding hidden vector to obtain the (i + 1)-th address label prediction result, and so on. Decode multiple times until the condition for ending the decoding is met and then terminate the decoding prediction. Specifically, the pre-trained address category prediction model decoder is used to predict the hierarchical categories of the address text, decode the address labels of each level in order from high to low of the category hierarchy, and combine the address labels of each level to obtain the final address category prediction result.

[0027] In this embodiment, the decoder performs multiple decodings on the second semantic encoding matrix output by the encoder. The multiple decoding process is performed iteratively in an autoregressive manner. Specifically, the input word for the first decoding is " <bos>"Use an embedding layer to obtain" <bos>The vector representation of "" is superimposed with the global hierarchical feature vector and combined with the second semantic encoding vector output by the encoder, and input into the Transformer Decoder to obtain the first decoded latent vector. Then, the first decoded latent vector is used by the classification layer to predict the first predicted address label "commercial residential". This is then used as the input word for the second decoding of the decoder, and the above operation is repeated to obtain the second predicted address label "industrial park". This is then used as the input word for the third decoding of the decoder, and the above operation is repeated to obtain the third predicted address label "industrial park". The address label predicted in the fourth decoding is the termination decoding marker. <eos>This concludes the decoding process. For details, please refer to [link / reference needed]. Figure 4 Schematic diagram of the decoder structure of the address category prediction model.

[0028] In this embodiment, the pre-trained address category prediction model structure includes an encoder, a decoder, and a graph encoder for global hierarchical extraction of the address label system. The encoder performs semantic encoding on the address text to obtain a semantic encoding vector. The decoder predicts the hierarchical category of the address text by sequentially decoding the address labels at each level in descending order of category hierarchy and merging the address labels at each level to obtain the final address category prediction result. The encoder model structure is constructed using the Encoder module of the Transformer model. The decoder model structure is constructed using the Decoder module of the Transformer model. Furthermore, the encoder, decoder, and hierarchical feature encoder model structures can also be constructed using other neural network structures.

[0029] Each decoding process involves address tag prediction, which is essentially a multi-label classification problem. The tag set consists of tags at various levels of the address tag system and four special categories: <bos> 、 <eos> 、 <pad> 、 <unk>.

[0030] The address labeling system has a tree-like hierarchical structure, and global hierarchical features of the address labeling system can be obtained using graph representation methods.

[0031] The decoding termination condition can be set to a fixed number of decoding attempts or the current decoding result as the end marker. <eos>.

[0032] S400. Combine the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result.

[0033] In S400 of this embodiment, the address label system is a tree-like hierarchical structure, and the global hierarchical features of the address label system are obtained by a graph representation-based method.

[0034] Specifically, in this embodiment, as follows: Figure 4 The address category prediction model decoder structure diagram is shown. The address label predicted by the first decoding is "commercial residential", the address label predicted by the second decoding is "industrial park", and the address label predicted by the third decoding is "industrial park". The results of the three decodings are combined to obtain the final address category prediction result "commercial residential-industrial park-industrial park".

[0035] A training method for an address category prediction model. This embodiment utilizes Amap (Gaode Maps) to crawl address data and obtain its category information. Address text and its corresponding address category are acquired as training data. The encoder of the address category prediction model semantically encodes each character in the address text, obtaining a semantic encoding matrix. The decoder of the address category prediction model performs an initial decoding prediction on the semantic encoding vector to obtain the first address tag of the address text. Then, the actual address tag of the address text is used as input for the next decoding attempt, and so on, decoding is repeated multiple times until the decoding termination condition is met. Finally, loss calculation is performed based on the multiple address tags obtained from the multiple decoding attempts and the actual address categories to optimize the address category prediction model.

[0036] This embodiment discloses a method for address text category prediction, comprising: acquiring the address text to be predicted; semantically encoding the address text to obtain a first semantic encoding vector; processing the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector; decoding the second semantic encoding matrix output by the encoder multiple times using a second preset rule to obtain multiple decoded address tags; and combining the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result. This invention implicitly introduces the type and boundary information of address elements by adding an attention mechanism to the encoder, and models the local and global relationships between hierarchical tags in the decoder, thereby improving the accuracy of the address category prediction model. Example 2

[0037] Based on the address text category prediction method of Embodiment 1, this embodiment discloses an address text category prediction system, including: an address text acquisition unit, a second semantic encoding vector acquisition unit, an address tag acquisition unit, and an address text category prediction result unit; wherein: The address text acquisition unit is used to acquire the address text to be predicted for address category. The second semantic encoding vector acquisition unit is used to perform semantic encoding on the address text to obtain the first semantic encoding vector, and to process the first semantic encoding vector using a first preset rule to obtain the second semantic encoding vector. The address tag acquisition unit is used to decode the second semantic encoding matrix output by the encoder multiple times using a second preset rule to obtain the address tags after multiple decodings. The address text category prediction result unit is used to combine the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result.

[0038] In this embodiment, the specific working methods of the address text acquisition unit, the second semantic encoding vector acquisition unit, the address tag acquisition unit, and the address text category prediction result unit have been described in detail in Embodiment 1, and will not be repeated here.

[0039] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0040] In the detailed description above, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features in a single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, with each claim representing a separate preferred embodiment of the invention.

[0041] Those skilled in the art will also understand that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments herein can be implemented as electronic hardware, computer software, or a combination thereof. To clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in alternative ways for each specific application; however, such implementation decisions should not be construed as departing from the scope of this disclosure.

[0042] The steps of the methods or algorithms described in conjunction with the embodiments herein can be directly embodied in hardware, software modules executed by a processor, or a combination thereof. The software modules can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in a user terminal. Alternatively, the processor and storage medium can exist as discrete components in the user terminal.

[0043] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. This software code can be stored in memory units and executed by a processor. The memory units can be implemented within the processor or outside the processor; in the latter case, they are communicatively coupled to the processor via various means, as is well known in the art.

[0044] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as it is understood when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."< / eos> < / unk> < / pad> < / eos> < / bos> < / eos> < / bos> < / bos> < / eos>

Claims

1. A method of address text category prediction, characterized by, include: S100. Obtain the address text to be predicted for address category; S200. Semantically encode each word of the address text to obtain a first semantic encoding vector. Process the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector. Combine the second semantic encoding vectors of all words in the address text to obtain a second semantic encoding matrix. In S200, the first preset rule includes: Construct an address type matrix, where each row of the address type matrix corresponds to an implicit address type vector, and the implicit address type vector contains type information of the address elements; An attention mechanism is introduced to calculate the correlation between the first semantic encoding vector and each implicit address type vector in the address type matrix, thereby generating a weight score matrix; The implicit address type vectors are weighted and summed according to their weight scores to obtain the second semantic encoding vector of implicit fused address element type information and boundary information; S300. The second semantic encoding matrix output by the encoder is decoded multiple times using the second preset rule to obtain the address tags after multiple decodings. In S300, the second semantic encoding matrix output by the encoder is decoded multiple times using the second preset rule to obtain the address tags after multiple decodings. The second preset rule includes: obtaining the address tag output by the i-th decoding through the input layer of the pre-trained address category prediction model decoder and vectorizing it using the embedding layer, and superimposing the global hierarchical features of the address tag system. Then, the second semantic encoding vector of the address text is combined with the decoding to obtain the (i+1)-th decoding latent vector. Then, the (i+1)-th decoding latent vector is predicted using the classification layer to obtain the (i+1)-th address tag prediction result. This process is repeated multiple times until the condition for ending the decoding is met, at which point the decoding prediction is terminated. S400. Combine the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result.

2. The method for predicting address text categories as described in claim 1, characterized in that, In S200, a pre-trained address category prediction model encoder is used to semantically encode the address text to obtain the first semantic encoding vector.

3. The method for predicting address text categories as described in claim 1, characterized in that, In S200, the input address text is split into word sequences, the word sequences are vectorized using an embedding layer, and then semantically encoded using the Transformer Encoder module to obtain the first semantic encoding vector of the address text.

4. The method for predicting address text categories as described in claim 1, characterized in that, The pre-trained address category prediction model decoder is used to predict the hierarchical category of the address text. It decodes the address tags of each level in descending order of category level, and merges the address tags of each level to obtain the final address category prediction result.

5. The method for predicting address text categories as described in claim 1, characterized in that, Decoding can be terminated by setting a fixed number of decoding attempts or by using the current decoding result as the end marker. <eos> 。< / eos> 6. The method for predicting address text categories as described in claim 1, characterized in that, In S400, the address label system is a tree-like hierarchical structure, and a graph-based representation method is used to obtain the global hierarchical features of the address label system.

7. A system for predicting address text categories, employing any one of the address text category prediction methods according to claims 1-6, characterized in that, include: The unit comprises an address text acquisition unit, a second semantic encoding vector acquisition unit, an address tag acquisition unit, and an address text category prediction result unit; wherein: The address text acquisition unit is used to acquire the address text to be predicted for address category. The second semantic encoding vector acquisition unit is used to perform semantic encoding on each word of the address text to obtain a first semantic encoding vector, process the first semantic encoding vector using a first preset rule to obtain a second semantic encoding vector, and combine the second semantic encoding vectors of all words in the address text to obtain a second semantic encoding matrix; the first preset rule includes: Construct an address type matrix, where each row of the address type matrix corresponds to an implicit address type vector, and the implicit address type vector contains type information of the address elements; An attention mechanism is introduced to calculate the correlation between the first semantic encoding vector and each implicit address type vector in the address type matrix, thereby generating a weight score matrix; The implicit address type vectors are weighted and summed according to their weight scores to obtain the second semantic encoding vector of implicit fused address element type information and boundary information; The address tag acquisition unit is used to decode the second semantic encoding matrix output by the encoder multiple times using a second preset rule to obtain address tags after multiple decodings. The second preset rule includes: obtaining the address tag output by the i-th decoding through the input layer of the pre-trained address category prediction model decoder and vectorizing it using an embedding layer, and superimposing the global hierarchical features of the address tag system. Then, it combines the second semantic encoding vector of the address text to decode and obtain the (i+1)-th decoding latent vector. Then, it uses a classification layer to predict the (i+1)-th decoding latent vector to obtain the (i+1)-th address tag prediction result. This process is repeated multiple times until the decoding end condition is met, at which point the decoding prediction is terminated. The address text category prediction result unit is used to combine the address tags obtained from multiple decodings according to the address tag system to obtain the final address category prediction result.

Images