Address text processing method, device, equipment and computer storage medium
By using occlusion identifiers and multi-head self-attention layer processing technology in the address text processing device, the problem of irregular administrative division information in the address text is solved, standardized processing and completion are achieved, processing efficiency is improved and manual intervention is reduced .
Patent Information
- Application Number
- CN201911046400.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-10-30
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2039-10-30
AI Technical Summary
In the prior art, the standardized processing of address texts has the problem of irregular and inconsistent address text expression, especially the incomplete administrative division information, which leads to the need for complex matching rules and a large number of manual interventions, which affects the processing efficiency.
The address text processing device including an input layer, a hidden layer and an output layer is adopted to process the context information in the address text through an omnidirectional communication connection and a multi-head self-attention layer, and to use an occlusion identifier to complete and standardize administrative division information.
It realizes standardized processing of address text, simplifies the maintenance of matching rules, reduces manual intervention, improves processing efficiency, and can accurately complete administrative division information and uniformly express the format.
Smart Images

Figure CN112749560B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to an address text processing method, device, equipment and computer storage medium. Background Art
[0002] In the prior art, the address texts used in daily life have problems such as non-standard, non-uniform and incomplete expressions, such as mixing of traditional and simplified Chinese characters, mixing of Chinese and Arabic characters, non-standard expressions, missing of some administrative divisions, etc.
[0003] For example, "Wenyi West Road, Yuhang District" only includes district-level administrative division information, and no provincial, municipal, etc. administrative division information. The Inner Mongolia in "Hohhot City, Inner Mongolia" is the abbreviation of the Inner Mongolia Autonomous Region. In order to facilitate subsequent operations such as navigation based on the address text, the address text needs to be standardized, such as administrative division completion and normalization operations. Completion operation refers to filling in the missing administrative divisions, and normalization operation is to unify different expressions of the same administrative division.
[0004] In the prior art, one way to process administrative divisions is to maintain a table containing all administrative divisions; when standardizing address text, perform word segmentation and administrative division marking operations on the input address text; and then perform administrative division completion and normalization operations by looking up the table. This standardized processing method requires the pre-fabrication of relatively complex matching rules, which makes it difficult to maintain matching rules and requires more manual intervention, affecting processing efficiency. Summary of the invention
[0005] In view of this, an embodiment of the present application provides an address text processing solution to solve some or all of the above problems.
[0006] According to a first aspect of an embodiment of the present application, there is provided an address text processing device, comprising: an input layer, the input layer comprising a plurality of input nodes, the input layer being used to receive an address text and encode the address text; a hidden layer, the hidden layer comprising a plurality of layers, each layer comprising a plurality of hidden layer nodes, the hidden layer being communicatively connected to the input layer, and processing the encoded information output by the input layer; an output layer, the output layer being communicatively connected to the hidden layer, and obtaining a processed address text according to the output of the hidden layer; wherein the input layer and the hidden layer are omnidirectionally connected, and the hidden layer nodes of two adjacent hidden layers are omnidirectionally connected; the hidden layer nodes are used to process the context information contained in the address text in the process of processing the encoded information.
[0007] According to the second aspect of the embodiment of the present application, an address text processing method is provided, comprising: using a masking identifier to complete the administrative division information of the address text to be processed to generate a completed address text; inputting the completed address text into the address text processing device described in the first aspect, and processing the completed address text by the address text processing device; obtaining a standard address text output by the address text processing device, wherein in the standard address text, the text identified by the masking identifier is replaced with standard administrative division information.
[0008] According to the third aspect of the embodiment of the present application, an address text processing device is provided, comprising: a placeholder completion module, used to use an obstruction identifier to perform administrative division completion on the address text to be processed, and generate a completed address text; a processing module, used to input the completed address text into the address text processing device described in the first aspect, and process the completed address text by the address text processing device; a standard address output module, used to obtain a standard address text output by the address text processing device, wherein in the standard address text, the text identified by the obstruction identifier is replaced with standard administrative division information.
[0009] According to a third aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the address text processing method as described in the second aspect is implemented.
[0010] According to the address text processing scheme provided in the embodiment of the present application, the address text processing device can process the address text. For address texts with irregular administrative division information, the existing administrative division data therein can be standardized; for address texts lacking administrative division information, the missing administrative division data in the address text can be supplemented, thereby achieving standardized processing of the address text and facilitating the subsequent use of the address text. It also solves the problem that the address text standardization processing method in the prior art requires the prefabrication of relatively complex matching rules, resulting in difficulty in maintaining the matching rules and requiring more manual intervention, thereby improving processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0012] Figure 1 It is a structural diagram of an address text processing device according to the first embodiment of the present application;
[0013] Figure 2 This is a schematic diagram of the structure of a hidden layer node of an address text processing device according to the second embodiment of the present application;
[0014] Figure 3A A flowchart of a method for processing an address text according to Embodiment 3 of the present application;
[0015] Figure 3B for Figure 3A A flowchart of a training process of an address text processing device in the illustrated embodiment;
[0016] Figure 4 It is a flowchart of a training method of an address text processing device in the address text processing method according to the fourth embodiment of the present application;
[0017] Figure 5 is a structural block diagram of an address text processing device according to Embodiment 5 of the present application;
[0018] Figure 6 is a structural block diagram of an address text processing device according to Embodiment 6 of the present application;
[0019] Figure 7 This is a schematic diagram of the structure of an electronic device according to Embodiment 7 of the present application. DETAILED DESCRIPTION
[0020] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.
[0021] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application. Example
[0022] Reference Figure 1 , shows a structural diagram of an address text processing device according to Embodiment 1 of the present application.
[0023] The address text processing device of this embodiment includes an input layer 101, a hidden layer 103 and an output layer 105. The input layer 101 includes a plurality of input nodes 1011, and the input layer 101 is used to receive the address text and encode the address text. The hidden layer 103 includes a plurality of layers, each layer includes a plurality of hidden layer nodes 1031, and the hidden layer 103 is connected to the input layer 101 in communication, and processes the encoded information output by the input layer 101. The output layer 105 is connected to the hidden layer 103 in communication, and obtains the processed address text according to the output of the hidden layer 103. Among them, the input layer 101 and the hidden layer 103 are connected in an omnidirectional communication, and the hidden layer nodes 1031 of two adjacent hidden layers 103 are connected in an omnidirectional communication; the hidden layer nodes 1031 are used to process the context information contained in the address text in the process of processing the encoded information.
[0024] The address file information may include administrative division data or may not include administrative division data, for example, "No. 56, Zhongguancun Street, Haidian District, Beijing", "Hohhot City, Inner Mongolia", "Yuhang District, Hangzhou City", "Laojuntang Village, Shibalidian Township", etc.
[0025] In this embodiment, the words in the address text may include only address words, or may include address words and shielding identifiers.
[0026] Among them, the address words can be understood as the elements constituting the address text, and the address words include but are not limited to: provincial elements, municipal elements, district elements, town elements, road elements, community elements, building number elements, floor elements and room elements. Among them, provincial elements, municipal elements, district elements and town elements are address words containing administrative division information. Of course, the classification of administrative divisions may be different in different application scenarios, and the address words containing administrative divisions may also be different, which is not limited in this embodiment.
[0027] Those skilled in the art may use any appropriate symbol and / or character as the mask identifier as needed, such as: “[mask]”, “***”, etc., which is not limited in this embodiment.
[0028] The input layer 101 includes a plurality of input nodes 1011, and each output node 1051 is used to receive a word in the address text and encode the word. The input layer 101 can encode the address text in an appropriate manner. For example, the words in the address text (such as address words) are encoded into word vectors and processed into corresponding word vectors.
[0029] For another example, the input layer 101 includes a word vector encoding layer and a position information encoding layer. The word vector encoding layer and the position information encoding layer each include a plurality of input nodes 1011, and the input nodes 1011 of the word vector encoding layer are used to encode the words in the received address text into corresponding word vectors. The input nodes 1011 in the position encoding layer are used to encode the word vector into an input word vector according to the position information of the word in the address text, and input the input word vector as the encoding information into the hidden layer 103.
[0030] The word vector encoding layer can encode words into corresponding word vectors in any appropriate manner, for example, using the Word2vec method, the one-hot method, the N-gram method, etc.
[0031] The hidden layer 103 includes multiple layers, each of which includes multiple hidden layer nodes 1031. Each hidden layer node 1031 can be a neuron. The hidden layer 103 is connected to the input layer 101 for communication and processes the coded information output by the input layer 101. The hidden layer node 1031 is used to process the context information contained in the address text in the process of processing the coded information. The context information contained in the address text is obtained by processing the coded information.
[0032] In this embodiment, the input layer 101 includes a plurality of input nodes 1011, and the hidden layer nodes 1031 are omnidirectionally connected to the input layer 101, and the hidden layer nodes 1031 of two adjacent hidden layers 103 are omnidirectionally connected to each other. That is, each hidden layer node 1031 is connected to all input nodes 1011. Any hidden layer node 1031 of the latter layer is connected to all hidden layer nodes 1031 of the previous layer. In this way, any hidden layer node 1031 can obtain all words in the address text, thereby ensuring that all context information can be obtained during the processing of the address text, and ensuring the accuracy of the processing. The processing of the coded information by the hidden layer 103 can be determined as needed, and this embodiment does not limit this.
[0033] The output layer 105 is connected to the hidden layer 103 for obtaining the processed address text according to the output of the connected hidden layer 103. In this embodiment, the output layer 105 includes a plurality of output nodes 1051, and each output node 1051 is connected to only one hidden layer node 1031. The output layer 105 can obtain the processed address text according to the output of the connected hidden layer 103 in any appropriate manner. For example, the output layer 105 can be a classifier, and obtains the processed address text by classification.
[0034] In this embodiment, the first M output nodes 1051 of the text processing device are used to output standard administrative division information, where M is a positive integer greater than 1. For example, in this embodiment, the first four output nodes 1051 are used to output standard administrative division information, corresponding to the provincial level, municipal level, district level, and town level, respectively.
[0035] Through this embodiment, the address text processing device can process the address text. For address texts with irregular administrative division information, the existing administrative division data can be standardized; for address texts lacking administrative division information, the missing administrative division data in the address text can be supplemented, thereby achieving standardized processing of the address text, making it convenient for the subsequent use of the address text. It also solves the problem that the address text standardization processing method in the prior art requires the pre-fabrication of relatively complex matching rules, resulting in difficulty in matching rule maintenance and requiring more manual intervention, thereby improving processing efficiency. Example
[0036] Reference Figure 2 , which shows a schematic diagram of the structure of the address text processing device of the second embodiment of the present application.
[0037] The address text processing device of this embodiment further optimizes the address text processing device in the first embodiment.
[0038] As described in the first embodiment, the address text processing device includes an input layer 101, a hidden layer 103 and an output layer 105. The hidden layer 103 includes multiple layers, each of which includes multiple hidden layer nodes 1031. Each of the hidden layer nodes 1031 includes a multi-head self-attention layer 1031a and a feedforward network layer 1031b.
[0039] The multi-head self-attention layer 1031a is used to extract context information from the coded information output by the input node 1011 corresponding to the current hidden layer node 1031 in the input layer 101 and the coded information output by other input nodes 1011 in the input layer 101. Figure 2 The hidden layer node 1031 in the first column of , its corresponding input node 1011 is the E1 node. Figure 2 The hidden layer node 1031 in the second column, its corresponding input node 1011 is the E2 node.
[0040] The multi-head self-attention layer 1031a can well extract the context information corresponding to the words in the address text that are mainly processed by the current hidden layer node 1031 from the encoded information. For example, the word input in the input node 1011E1 corresponding to the hidden layer node 1031 in the first column is "Beijing", and the hidden layer node 1031 in the first column is used to process the context information of each word in the address text and "Beijing". Therefore, the standard administrative division information corresponding to "Beijing" can be well predicted based on the context information to ensure the processing effect.
[0041] The feedforward network layer 1031b is used to receive and process the context information output by the corresponding multi-head self-attention layer 1031a. For example, the context information is subjected to dimensionality reduction processing, etc. Depending on the needs, the feedforward network layer 1031b can process the context information differently, which is not limited in this embodiment.
[0042] Optionally, each of the hidden layer nodes 1031 further includes a first normalization layer 1031c and / or a second normalization layer 1031d.
[0043] The first normalization layer 1031c is communicatively connected between the multi-head self-attention layer 1031a and the feedforward network layer 1031b, normalizes the context information output by the multi-head self-attention layer 1031a, and transmits the normalization result to the feedforward network layer 1031b.
[0044] Those skilled in the art may normalize the context information in an appropriate manner as needed, and this embodiment does not limit this.
[0045] After the second normalization layer 1031d is communicatively connected to the feedforward network layer 1031b, the data output by the feedforward network layer 1031b is normalized. Those skilled in the art can perform normalization on the data output by the feedforward network layer 1031b as needed, and this embodiment does not limit this. The normalization processing methods of the first normalization layer 1031c and the second normalization layer 1031d can be the same or different.
[0046] Optionally, in this embodiment, the address text processing device receives a call from an input method application, receives the address text sent by the input method application through the input layer 101, and after processing by the input layer 101 and the hidden layer 103, outputs the converted standard address text to the input method application through the output layer 105 and displays it through the input method application.
[0047] The address text processing device may provide a calling interface (API), and the input method application transmits the address text to the input layer 101 of the address text processing device by calling the calling interface. After receiving the address text, the input layer 101 encodes it and inputs the encoded information of the encoding process to the hidden layer 103 for processing. The output layer 105 is used to output the processed standard address text obtained according to the output of the hidden layer 103 to the input method application, and display it through the input method application.
[0048] For example, the user inputs "Zhongguancun East Road, Haidian District" through the input method application, and the address text processing device processes "Zhongguancun East Road, Haidian District" into "Zhongguancun East Road, Haidian District, Beijing" according to the call of the input method, and outputs it to the input method application. In this way, the user can quickly input the standard address text, which not only ensures the accuracy of the user's address filling, but also eliminates the need for the user to perform tedious input.
[0049] Through this embodiment, the address text processing device can process the address text. For address texts with irregular administrative division information, the existing administrative division data can be standardized; for address texts lacking administrative division information, the missing administrative division data in the address text can be supplemented, thereby achieving standardized processing of the address text, making it convenient for the subsequent use of the address text. It also solves the problem that the address text standardization processing method in the prior art requires the pre-fabrication of relatively complex matching rules, resulting in difficulty in matching rule maintenance and requiring more manual intervention, thereby improving processing efficiency. Example
[0050] Reference Figure 3A , shows a step flow chart of an address text processing method according to embodiment three of the present application.
[0051] The address text processing method of this embodiment includes:
[0052] Step S10: Use the mask identifier to complete the administrative division information of the address text to be processed to generate a completed address text.
[0053] Step S20: inputting the completed address text into the above-mentioned address text processing device, and processing the completed address text by the address text processing device.
[0054] Step S30: obtaining a standard address text output by the address text processing device, wherein in the standard address text, the text identified by the obstruction identifier is replaced with standard administrative division information.
[0055] Those skilled in the art may use any appropriate symbol and / or character as a mask identifier, such as "[mask]", "***", etc., as needed, and this embodiment does not limit this. For example, if the address text is "No. 56, Zhongguancun Street, Beijing", the address text after the administrative division information is supplemented with the mask identifier may be "No. 56, Zhongguancun Street, Beijing [mask]".
[0056] The completed address text is input as the address text into the address text processing device, so that the address text is processed by the device and a standard address text is output. In the standard address text, the text identified by the obstruction identifier is replaced with standard administrative division information, such as "No. 56, Zhongguancun Street, Haidian District, Beijing".
[0057] This method can be used to standardize address texts, so that standard address texts can be used later to improve reliability and accuracy.
[0058] Optionally, the address text to be processed is a call to receive an input method application, and the address text to be processed sent by the input method application is received through the input layer. The method also includes: outputting the converted standard address text to the input method application through the output layer and displaying it through the input method application.
[0059] For example, a user inputs the address text "East Zhongguancun Road, Beijing" through an input method application. After receiving the address text, the input method application calls an address text processing device, which processes the address text and converts it into a standard address text "East Zhongguancun Road, Haidian District, Beijing", and then sends it to the input method application. After receiving the converted standard address text, the input method application displays the standard address text through a candidate word interface for the user to select.
[0060] By receiving the call of the input method application, the address text to be processed sent by the input method application can be obtained. The address text processing device executes the aforementioned steps to process the address text to be processed into a standard address text containing standard administrative division information, and transmits the standard address text to the input method application for display. In this way, the user can conveniently input the standard address text through the input method, which makes it convenient for the user to use and can ensure the accuracy of the input address text.
[0061] In this embodiment, before using the occluded identifier to complete the administrative division information of the address text to be processed, the method further includes: performing standard division training on the address text processing device.
[0062] In one feasible manner, the address text processing device is trained with standard divisions, such as Figure 3B As shown, the following steps may be included:
[0063] Step S102: performing a first masking operation on at least part of the address words of the first address text, generating a pre-training sample according to the first masking result, and using the pre-training sample to pre-train an address text processing device for address text processing.
[0064] In this embodiment, the address text processing device may be a bidirectional language model.
[0065] The first address text may be a general address text, which may include administrative division data or may not include administrative division data, for example, "No. 56, Zhongguancun Street, Haidian District, Beijing", "Hohhot City, Inner Mongolia", "Yuhang District, Hangzhou City", "Laojuntang Village, Shibalidian Township", etc.
[0066] Address words can be understood as elements constituting the first address text, and address words include but are not limited to: provincial elements, municipal elements, district elements, town elements, road elements, community elements, building number elements, floor elements, and room elements, etc. Among them, provincial elements, municipal elements, district elements, and town elements are address words containing administrative division information.
[0067] Of course, the hierarchy of administrative divisions may be different in different application scenarios, and the address words containing administrative divisions may also be different, which is not limited in this embodiment.
[0068] The first masking operation can be implemented as a replacement operation, a deletion operation, etc. For example, a preset masking identifier is used to replace at least part of the address words that need to be masked. Those skilled in the art can use any appropriate symbol and / or character as the masking identifier as needed, such as: "[mask]", "***", etc., which is not limited in this embodiment.
[0069] For example, the first address text is: "No. 56, Zhongguancun Street, Haidian District, Beijing". If the first masking operation is performed on the address word "Haidian District", the corresponding first masking result is: "No. 56, Zhongguancun Street, ***, Beijing".
[0070] The pre-training sample is generated according to the first shielding result, and is used to pre-train the address text processing device. During pre-training, a more common and easier to obtain first address text can be used to generate the pre-training sample, which can reduce the collection and processing cost of the training sample.
[0071] According to different needs, those skilled in the art may adopt address text processing devices with different structures for address text processing, and this embodiment does not limit this.
[0072] For example, in this embodiment, the address text processing device includes an input layer, multiple hidden layers and an output layer, and each hidden layer includes multiple parallel neurons. The hidden layer receives data from the input layer connected to it and is used to process the received data. Depending on the processing required, the neurons in the hidden layer can be different. The output layer is used to receive data from the hidden layer connected to it and output a prediction result based on the received data.
[0073] When pre-training the address text processing device, the pre-training sample can be input into the address text processing device, and the output data of the address text processing device can be obtained. The output data can be understood as the address word corresponding to the occluded identifier predicted by the address text processing device according to the input pre-training sample. According to the difference between the output data and the occluded address word in the original first address text, the training parameters in the address text processing device are adjusted, so that the address text processing device learns better training parameters, and the pre-training of the address text processing device is completed.
[0074] The training parameters include but are not limited to conventional training parameters such as weight parameters and bias parameters in a supervised learning model.
[0075] Step S104: performing a second masking operation on part of the administrative division data of the second address text for training, and generating a target task training sample according to the second masking result.
[0076] The second address text is an address text corresponding to the target task. For example, if the target task is to complete and unify the administrative divisions in the address text, the second address text is an address text containing at least part of the administrative division data. Of course, the second address text used for different target tasks may be different.
[0077] The second shielding operation may be the same as or different from the first shielding operation. When the second shielding operation is the same as the first shielding operation and both are replacement operations, it may use the same shielding identifier as the first shielding operation or a different shielding identifier from the first shielding operation.
[0078] In this embodiment, the occlusion identifier used in the second occlusion operation is the same as the occlusion identifier used in the first occlusion operation, which helps to make full use of the training parameters learned by the address text processing device during the pre-training process, so that the subsequent training effect for the target task is better.
[0079] When the second masking operation is different from the first masking operation, the second masking operation may be a deletion operation. For example, if the second address text is "No. 56, Zhongguancun Street, Haidian District, Beijing", after performing the second masking operation on part of the administrative division data such as "Haidian District", the second masking result may be "No. 56, Zhongguancun Street, Beijing". The corresponding target task training sample may be generated according to the second masking result.
[0080] Step S106: Use the target task training sample to perform target training on the pre-trained address text processing device, so as to generate standard administrative division information through the trained address text processing device.
[0081] Among them, in the target training process, the training parameters of the first N hidden layers in the address text processing device remain unchanged as the parameters obtained through the pre-training, and the training parameters of other hidden layers in the address text processing device are trained, and N is a positive integer greater than or equal to 1.
[0082] Among them, N can be appropriately set by those skilled in the art according to actual needs in specific applications. For example, assuming that the address text processing device includes 10 hidden layers, in the target training process, the training parameters of the first 8 hidden layers can be determined as the parameters obtained by pre-training, and the training parameters of the first 8 hidden layers are kept unchanged, and the training parameters of the last 2 hidden layers are trained using the target task training samples.
[0083] Of course, according to the difference between the pre-training samples used in pre-training and the target task training samples, the number of hidden layers that keep the training parameters unchanged can be appropriately adjusted. For example, the smaller the difference between the two, the more hidden layers that keep the training parameters unchanged can be, and vice versa, the fewer hidden layers that keep the training parameters unchanged can be.
[0084] Since the address text processing device has been pre-trained through pre-training samples, the address text processing device has learned training parameters that have better processing effects on general address texts. Therefore, when training for the target task, some of the parameters obtained by pre-training can be used, thereby reducing the number of target task training samples and training time required for training the target task, improving training efficiency, reducing training costs, and ensuring better training effects.
[0085] In addition, when the target task changes, for example, the target task changes from completing administrative division information and unifying the expression format to other tasks, appropriate new target task training samples can be selected according to the new target task, and the new target task training samples can be used for training, so as to quickly obtain an address text processing device that meets the needs. This can reduce the training cost when the target task changes and improve the training efficiency.
[0086] The trained address text processing device can process the address text, accurately determine the standard administrative division information missing in the address text, and convert the administrative division data that exists in the address text but does not conform to the standard expression format into standard administrative division information, achieving good address text processing effects and laying a good foundation for the subsequent use of the address text.
[0087] In this embodiment, a pre-training sample for pre-training the address text processing device is obtained by performing a first masking operation on the first address text, and then the address text processing device is pre-trained. In this way, the difficulty of obtaining pre-training samples is reduced while ensuring that the address text processing device can learn better training parameters. In the target training process, a second address text corresponding to the target task is used to perform a second masking operation, and a target task training sample is generated according to the second masking result. When the address text processing device is trained using the target task training sample, only the training parameters of the hidden layers in the address text processing device except the first N hidden layers are trained, which improves the training efficiency, and a small amount of target task training samples can be used to achieve a good training effect, reducing the difficulty of target task training. The trained address text processing device can complete the missing administrative division data in the address text and standardize the existing administrative division data therein, which solves the problem that the standardized processing method in the prior art needs to pre-formulate more complex matching rules, resulting in difficulty in matching rule maintenance and requiring more manual intervention, thereby improving the processing efficiency.
[0088] The address text processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a tablet computer, a mobile phone, etc.) and a PC, etc. Example
[0089] First, refer again Figure 3A The address text processing method comprises: step S10: using the masking identifier to complete the administrative division information of the address text to be processed to generate the completed address text. Step S20: inputting the completed address text into the above-mentioned address text processing device, and processing the completed address text by the address text processing device. Step S30: obtaining the standard address text output by the address text processing device, wherein in the standard address text, the text identified by the masking identifier is replaced with the standard administrative division information.
[0090] Different from the third embodiment, this embodiment uses a specific address text processing device as an example to further illustrate the use of the address text device in the third embodiment. The address text processing device example of this embodiment is a bidirectional language model, which is specifically described as follows.
[0091] See again Figure 1 , showing a structural diagram of the bidirectional language model. The bidirectional language model includes a plurality of parallel processing units, each of which includes an input node 1011 in the input layer 101 described in the first embodiment, a plurality of hidden layer nodes 1031 in a row in the hidden layer, and an output node 1051 in the output layer 105.
[0092] The number of processing units in the bidirectional language model is determined according to the maximum number of address words in the pre-training samples and the target task training samples. Address words refer to words containing address information in the first address text and the second address text. Different address word segmentation methods can be used to segment address words from the first address text and the second address text according to different needs. For example, the first address text is "No. 56, Zhongguancun East Road, Haidian District, Beijing", and the segmented address words are "Beijing City", "Haidian District", "Zhongguancun East Road" and "No. 56".
[0093] For each processing unit, the input layer is connected to the hidden layers in multiple processing units, and the output layer is connected to the hidden layer in the processing unit. In order to enable the two-way language model to correctly output standard administrative division information at all levels, the output layers of the first M processing units of the two-way language model are used to output standard administrative division information, where M is a positive integer greater than 1.
[0094] For example, a classifier is set in the output layer of the first M processing units to classify the address text and determine its corresponding standard administrative division information. The specific value of M can be determined as needed. For example, for address text that needs to output 4-level administrative divisions, the value of M is 4. The first processing unit is used to output provincial standard administrative division information, the second processing unit is used to output municipal standard administrative division information, the third processing unit is used to output district-level administrative division information, and the fourth processing unit is used to output town-level standard administrative division information.
[0095] Optionally, in order to improve the learning effect of the bidirectional language model on the deep semantic features in the address text, the structure of the neurons in the hidden layer included in each processing unit can be the structure of the Transformer model. The structure of the Transformer model is as follows: Figure 2As shown, it includes multiple encoders connected in sequence, each encoder includes a multi-head attention layer 1031a and a feedforward neural network layer 1031b. After the data is input into the Transformer model, it is processed by the multi-head attention layer (Multi-headattention), and then summed and normalized (add&norm), and then enters the feedforward neural network layer (feed forword) for processing. The data processed by the feedforward neural network layer is summed and normalized (add&norm) to obtain the output data of the encoder.
[0096] Of course, those skilled in the art may adopt other model structures to implement neurons in the hidden layers of each processing unit as needed, and this embodiment does not limit this.
[0097] The training method based on the aforementioned bidirectional language model includes the following steps:
[0098] Step S202: performing a first masking operation on at least part of the address words in the first address text, generating a pre-training sample according to the first masking result, and using the pre-training sample to pre-train a bidirectional language model for address text processing.
[0099] As described in Example 1, the first address text may be a general address text, which may include administrative division data or may not include administrative division data. The process of generating pre-training samples according to the first address text may be the same as the process described in Example 1, so it will not be repeated here.
[0100] The process of pre-training a bidirectional language model consists of the following steps:
[0101] Step A: Generate a first pre-trained word vector corresponding to each address word according to each address word in the pre-trained sample and its position information in the first address text, and generate a second pre-trained word vector corresponding to the occlusion identifier according to the occlusion identifier and its position information in the first address text.
[0102] For example, the first address text is "No. 56, Zhongguancun Street, Haidian District, Beijing", and its corresponding first masking result is "No. 56, Zhongguancun Street, Beijing [mask]", and the corresponding address words in the pre-trained sample are "Beijing", "[mask]", "Zhongguancun Street" and "No. 56". The semantic word vector corresponding to each address word is calculated using a word vector algorithm (such as a Word embedding algorithm) according to each address word, and then the position word vector corresponding to each address word is determined according to the position information of each address word in the first address text, and the semantic word vector and the position word vector of each address word are concatenated to obtain the first pre-trained word vector corresponding to each address word.
[0103] The process of generating the second pre-trained word vector is the same as the process of generating the first pre-trained word vector, so it will not be repeated here.
[0104] For a first address text in which the number of address words is less than the number of processing unit columns of the bidirectional language model, placeholder characters may be used to process the number of address words of the first address text to be the same as the number of processing unit columns.
[0105] Step B: Input each first pre-trained word vector or the second pre-trained word vector into the input layer of different processing units respectively, and distribute each first pre-trained word vector or the second pre-trained word vector to the hidden layer and output layer of each processing unit through the input layer of different processing units, so that each processing unit processes the first pre-trained word vector and the second pre-trained word vector.
[0106] like Figure 1 As shown, each first pre-trained word vector ( Figure 1 E1, E3~En) are input into the input layer of the 1st, 3rd to nth processing units one by one. The second pre-trained word vector ( Figure 1 E2) is input into the input layer of the second processing unit.
[0107] Through each input layer, each first pre-trained word vector and second pre-trained word vector are distributed to each hidden layer, processed by neurons in the hidden layer, and the processing results are input to the output layer connected to the hidden layer, and the output layer outputs the predicted address word.
[0108] Step C: using the blocked address words in the first address text as annotation data, and adjusting the training parameters in the bidirectional language model according to the difference between the annotation data and the predicted address words.
[0109] For example, calculations are performed based on the annotated data, a preset loss function, and the predicted address words to determine the difference between the annotated data and the predicted address words, and training parameters in the bidirectional language model are adjusted based on the difference.
[0110] Step D: Return to step A and continue training the bidirectional language model after adjusting the training parameters until the pre-training termination condition is met.
[0111] The pre-training termination condition can be set as needed, for example, the number of training times reaches a preset number or the difference between the labeled data and the predicted address word is less than a preset difference value.
[0112] Step S204: performing a second masking operation on part of the administrative division data of the second address text, and generating a target task training sample according to the second masking result.
[0113] The step S204 includes the following sub-steps:
[0114] Sub-step S2041: using a blocking identifier to replace part of the address words indicating administrative division data in the second address text.
[0115] As described in the first embodiment, the occlusion identifier used in the second occlusion operation may be the same as or different from the occlusion identifier used in the first occlusion operation.
[0116] In this embodiment, an identifier that is the same as the mask identifier used in the first mask operation, such as "[mask]", is used as an example. When performing the second mask operation, part of the address words indicating administrative division data in the second address text is replaced with the mask identifier. For example, if the second address text is "Xingyuan International, No. 24, Zhongguancun East Road, Haidian District, Beijing", and the address word "Beijing" is replaced with "[mask]", the replaced second address text is "[mask] Xingyuan International, No. 24, Zhongguancun East Road, Haidian District".
[0117] Sub-step S2042: taking the second address text replaced with the occlusion identifier as the second occlusion result, and generating a target task training sample according to the second occlusion result.
[0118] The second address text after replacement is the second masking result. When generating the target task training sample based on the second masking result, the second masking result can be segmented to obtain each address word in the second masking result, and the target task training sample can be generated based on these address words. For example, if the second masking result is: "[mask] No. 24, Zhongguancun East Road, Haidian District, Xingyuan International", the generated target task training sample can be "{{[mask]},{Haidian District},{Zhongguancun East Road},{No. 24},{Xingyuan International}}".
[0119] Of course, in other embodiments, any other appropriate method may be used to generate a target task training sample according to the second occlusion result, and this embodiment does not limit this.
[0120] Step S206: using the target task training sample to perform target training on the pre-trained bidirectional language model, so as to generate standard administrative division information through the trained bidirectional language model.
[0121] Wherein, during the target training process, the training parameters of the first N hidden layers in the bidirectional language model remain unchanged as obtained through the pre-training, and the training parameters of other hidden layers in the bidirectional language model are trained, where N is a positive integer greater than or equal to 1.
[0122] Assuming that the bidirectional language model includes 10 hidden layers, in the target training process, the training parameters of the first 8 hidden layers can be determined as the parameters obtained by pre-training, and the training parameters of the first 8 hidden layers are kept unchanged, and the training parameters of the last 2 hidden layers are trained using the target task training samples.
[0123] Of course, according to the difference between the pre-training samples used in pre-training and the target task training samples, the number of hidden layers that keep the training parameters unchanged can be appropriately adjusted. For example, the smaller the difference between the two, the more hidden layers that keep the training parameters unchanged can be, and vice versa, the fewer hidden layers that keep the training parameters unchanged can be.
[0124] During the target training process, the training parameters of other hidden layers can be randomly initialized.
[0125] In one feasible manner, the step S206 includes the following sub-steps:
[0126] Sub-step S2061: Generate a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier through the input node of the bidirectional language model according to each address word and corresponding position information, occlusion identifier and corresponding position information in the target task training sample.
[0127] For example, if the second address text is "Yuhang Town, Hangzhou, Zhejiang Province", the corresponding second masking result is "[mask] Yuhang Town, Hangzhou", and the corresponding address words in the target task training sample are "[mask]", "Hangzhou" and "Yuhang Town". The semantic word vector corresponding to each address word is calculated using a word vector algorithm (such as Word embedding) according to each address word, and then the position word vector corresponding to each address word is determined according to the position information of each address word in the second address text, and the semantic word vector and the position word vector of each address word are concatenated to obtain the first input word vector corresponding to each address word.
[0128] The process of generating the second input word vector is the same as the process of generating the first input word vector, so it will not be repeated here.
[0129] For a second address text in which the number of address words is less than the number of processing unit columns of the bidirectional language model, placeholder characters may be used to process the number of address words of the second address text to be the same as the number of processing unit columns.
[0130] Sub-step S2062: input the first input word vector or the second input word vector generated by the input node into the hidden layer and the output layer of different processing units respectively, so that each processing unit processes the first input word vector and the second input word vector.
[0131] like Figure 1As shown, the second input word vector ( Figure 1 E1 in the figure is input to the input layer of the first processing unit. The first input word vector ( Figure 1 In the figure, E2~En) are input one by one to the input layer of the 2nd to nth processing units.
[0132] The corresponding first input word vector or the second input word vector is distributed to the hidden layer of each processing unit of the bidirectional language model through the input layer of the different processing units.
[0133] like Figure 1 As shown, for a neuron in the first hidden layer in a bidirectional language model (denoted as neuron A), the input data of neuron A is the input word vector obtained by its corresponding input layer and the first input word vectors and / or second input word vectors distributed by other input layers.
[0134] For a neuron (denoted as neuron B) in a hidden layer that is neither the first nor the last layer, the input data of neuron B is the output data of each neuron in the previous hidden layer, and the output data of neuron B will be input into each neuron in the next hidden layer.
[0135] For the neurons in the last hidden layer (denoted as neuron C), the input data of neuron C is the output data of each neuron in the previous hidden layer, and the output data of neuron C is input into the output layer connected to it.
[0136] Each neuron in the hidden layer is used to process the input data to obtain semantic data between the input word vector obtained by its corresponding input layer and the distributed first input word vectors and / or second input word vectors.
[0137] Sub-step S2063: Determine the standard administrative division prediction information corresponding to the occlusion identifier according to the processing results of each processing unit.
[0138] In one feasible manner, the sub-step S2063 includes the following sub-steps:
[0139] Sub-step S2063a: Obtain the processing result of the processing unit where the input node receiving the second input word vector is located.
[0140] The processing unit where the input layer receiving the second input word vector is located can be understood as the processing unit that inputs the second input word vector (referred to as processing unit A). Correspondingly, the processing result of the processing unit where the input layer receiving the second input word vector is located is the data output by the last hidden layer of processing unit A.
[0141] Sub-step S2063b: input the obtained processing result into the classifier of the output node of the processing unit, and obtain the classification result of the classifier, and determine the standard administrative division prediction information corresponding to the occlusion identifier according to the classification result and the administrative division dictionary.
[0142] In this embodiment, the output layer includes a classifier, and the obtained processing results are input into the classifier, which performs classification and obtains classification results. The classifier can be a binary classifier or a multi-element classifier. For example, the classifier is a softmax classifier.
[0143] The standard administrative division prediction information corresponding to the occlusion identifier is determined from a preset administrative division dictionary according to the classification result.
[0144] The administrative division dictionary is determined based on the address text in the collected address library. Each dictionary word in the administrative division dictionary is standard administrative division information. This ensures that the bidirectional language model outputs standard administrative division information during target training, thereby ensuring the training effect of the bidirectional language model at the training site.
[0145] Step S2064: adjusting the training parameters of the other hidden layers except the first N hidden layers in the bidirectional language model according to the difference between the standard administrative division prediction information and the standard administrative division annotation information corresponding to the blocked address word.
[0146] After obtaining the standard administrative division prediction information, the difference between the standard administrative division prediction information and the standard administrative division annotation information corresponding to the obscured address words is determined based on the standard administrative division prediction information, the standard administrative division annotation information corresponding to the obscured address words and a preset loss function, and the training parameters of the other hidden layers in the bidirectional language model except the first N hidden layers are adjusted based on the difference.
[0147] The standard administrative division annotation information is determined based on the standard administrative division information annotation corresponding to the obscured address word in the second address text. For example, if the second address text is "Yuhang District, Hangzhou, Zhejiang Province", and the obscured address word is "Hangzhou", the corresponding standard administrative division annotation information is "Hangzhou City".
[0148] Step S2065: Return to step S2061 to continue to execute, so as to continue to train the bidirectional language model after adjusting the training parameters until the training termination condition is met.
[0149] After adjusting the training parameters of other hidden layers, return to step S2061 to continue to train the two-way language model after adjusting the training parameters. The training termination condition may be that the number of training times meets the preset number of training times, or the difference between the standard administrative division prediction information and the standard administrative division annotation information corresponding to the obscured address word is less than a preset difference value, etc.
[0150] After the bidirectional language model is trained, the address text can be input into the bidirectional language model, which processes it and outputs the address text containing standard administrative division information with complete administrative division data and unified expression format, thereby achieving the purpose of completing and unifying the administrative division information in the address text.
[0151] Step S208: Process the address text to be processed by using the trained bidirectional language model, and determine the standard administrative division information corresponding to the address text to be processed according to the processing result.
[0152] For example, if the address text to be processed is "Nanyuan Street, No. 2, Donghu South Road, Hangzhou, Zhejiang Province", the address words therein are "Zhejiang Province", "Hangzhou", "Donghu South Road", "No. 2" and "Nanyuan Street" respectively.
[0153] An optional process of processing the address text through a trained bidirectional language model includes the following steps:
[0154] Step E: Use placeholder characters to process the number of address words in the address text to be the same as the number of columns of the processing unit of the bidirectional language model.
[0155] Those skilled in the art may select any appropriate character as a placeholder character, such as “#”, etc., as needed, and this embodiment does not limit this.
[0156] Step F: Generate a corresponding input word vector based on each address word and its position information.
[0157] It should be noted that in this embodiment, for the convenience of description, the placeholder characters and the original address words in the address text are collectively referred to as address words. The method of generating the input word vector can be the same as the method of generating the first input word vector mentioned above, so it will not be repeated here.
[0158] Step G: Process each input word vector through a bidirectional language model, and determine the standard administrative division information based on the processing results.
[0159] Each input word vector is input into different processing units of the bidirectional language model, so that the hidden layer of each processing unit processes the input word vector. The processing result of the hidden layer is then input into the output layer connected thereto. Since the amount of standard administrative division information is limited, the classifier of the output layer can be used to classify each processing result to determine the standard administrative division information corresponding to the address text to be processed, thereby achieving administrative division completion and unified expression format of the address text.
[0160] For example, after processing the aforementioned address text to be processed, the standard administrative division information determined includes: "Zhejiang Province", "Hangzhou City" and "Yuhang District". Among them, "Hangzhou City" is the standardization of the expression format of the address word "Hangzhou" in the address text to be processed, and "Yuhang District" is the completion of the administrative division missing in the address text to be processed.
[0161] In this way, we can use the bidirectional language model and deep learning methods to learn the characteristics of the address text, so as to automatically determine the administrative division data therein and output the standard administrative division information corresponding to the administrative division data, so as to achieve the effect of automatic completion and normalization of the administrative division data.
[0162] Through this embodiment, a pre-training sample for pre-training a bidirectional language model is obtained by performing a first masking operation on the first address text, and then the bidirectional language model is pre-trained. In this way, the difficulty of obtaining pre-training samples is reduced while ensuring that the bidirectional language model can learn better training parameters. In the target training process, a second address text corresponding to the target task is used to perform a second masking operation, and a target task training sample is generated according to the second masking result. When the target task training sample is used to train the bidirectional language model, only the training parameters of the hidden layers other than the first N hidden layers in the bidirectional language model are trained, which improves the training efficiency, and a small amount of target task training samples can be used to achieve a good training effect, reducing the difficulty of target task training. The trained bidirectional language model can be used to complete the missing administrative division data in the address text, and the existing administrative division data therein can be standardized, which solves the problem that the standardized processing method in the prior art needs to pre-formulate more complex matching rules, resulting in the difficulty of matching rule maintenance and the need for more manual intervention, thereby improving the processing efficiency.
[0163] In addition, compared with the method of looking up tables to complete administrative divisions and the method of using shallow RNN structure text encoding to predict and normalize administrative divisions, the address text processing method of this embodiment has the advantages of not needing to formulate complex matching rules and reducing manual intervention, and pre-trained samples are easier to obtain and modify. When the target task changes, fewer target task training samples can be used to train the bidirectional language model, thereby obtaining a new bidirectional language model that meets the target task requirements. In addition, the model's contextual semantic understanding of the address text is improved through a deep bidirectional language model, so that the completion and normalization of administrative divisions at the output layer are better than the simple shallow RNN model, which can solve the problem that the shallow RNN model cannot effectively utilize the bidirectional context information in the address text, and directly maps the address text to a vector, making it easy for the non-administrative division part at the end of the text to affect the prediction result of the administrative division part during decoding, resulting in unstable output.
[0164] The address text processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a tablet computer, a mobile phone, etc.) and a PC, etc. Example
[0165] Reference Figure 5 , shows a structural block diagram of an address text processing device according to Embodiment 5 of the present application.
[0166] The address text processing device of this embodiment includes: a placeholder completion module 502, which is used to use the obstruction identifier to complete the administrative division of the address text to be processed to generate the completed address text; a processing module 504, which is used to input the completed address text into the above-mentioned address text processing device, and process the completed address text by the address text processing device; a standard address output module 506, which is used to obtain the standard address text output by the address text processing device, wherein in the standard address text, the text identified by the obstruction identifier is replaced by standard administrative division information.
[0167] The address text processing device can process the address text. For address texts with irregular administrative division information, the existing administrative division data can be standardized; for address texts lacking administrative division information, the missing administrative division data in the address text can be supplemented, thereby achieving standardized processing of the address text and facilitating the subsequent use of the address text. It also solves the problem in the prior art that the standardization processing method of address text requires the prefabrication of relatively complex matching rules, resulting in difficulty in maintaining the matching rules and requiring more manual intervention, thereby improving processing efficiency. Example
[0168] Reference Figure 6 , shows a structural diagram of an address text processing device according to embodiment six of the present application.
[0169] The address text processing device of this embodiment includes: a placeholder completion module 602, which is used to use the obstruction identifier to complete the administrative division of the address text to be processed to generate the completed address text; a processing module 604, which is used to input the completed address text into the above-mentioned address text processing device, and process the completed address text through the address text processing device; a standard address output module 606, which is used to obtain the standard address text output by the address text processing device, wherein in the standard address text, the text identified by the obstruction identifier is replaced with standard administrative division information.
[0170] Optionally, the device further comprises: a training module 600, configured to perform standard division training on the address text processing device.
[0171] Optionally, the training module 600 includes: a first masking module 6001, which is used to perform a first masking operation on at least part of the address words of the first address text for training, generate pre-training samples according to the first masking result, and use the pre-training samples to pre-train the address text processing device; a second masking module 6002, which is used to perform a second masking operation on part of the administrative division data of the second address text for training, and generate target task training samples according to the second masking result; a target training module 6003, which is used to perform target training on the pre-trained address text processing device using the target task training samples, so as to generate standard administrative division information through the trained address text processing device; wherein, during the target training process, the training parameters of the first N hidden layers in the address text processing device remain unchanged as obtained through the pre-training, and the training parameters of other hidden layers in the address text processing device are trained, and N is a positive integer greater than or equal to 1.
[0172] Optionally, the second masking module 6002 includes: a first replacement module, used to replace some address words in the second address text indicating administrative division data with a masking identifier; a generation module, used to use the second address text replaced with the masking identifier as the second masking result, and generate a target task training sample based on the second masking result.
[0173] Optionally, the first occlusion module 6001 is also used to generate, through the input node of the address text processing device, a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier according to each address word and the corresponding position information, the occlusion identifier and the corresponding position information in the target task training sample; distribute the first input word vector or the second input word vector generated by the input node to the hidden layer and output layer of different processing units, so that each processing unit processes the first input word vector and the second input word vector; determine the standard row corresponding to the occlusion identifier according to the processing results of each processing unit. Administrative division prediction information; adjusting the training parameters of the other hidden layers in the address text processing device except the first N hidden layers according to the difference between the standard administrative division prediction information and the standard administrative division labeling information corresponding to the blocked address words; returning to the input node of the address text processing device, and continuing to execute the step of generating a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier according to each address word and the corresponding position information, the occlusion identifier and the corresponding position information in the target task training sample, so as to continue to train the address text processing device after adjusting the training parameters until the training termination condition is met.
[0174] Optionally, the target training module 6003 is also used to obtain the processing result of the processing unit where the input node receiving the second input word vector is located; input the obtained processing result into the classifier of the output node of the processing unit, and obtain the classification result of the classifier, and determine the standard administrative division prediction information corresponding to the occlusion identifier based on the classification result and the administrative division dictionary.
[0175] Optionally, the address text to be processed is a call to receive an input method application, and the address text to be processed sent by the input method application is received through the input layer. The device also includes: a display module 608, which is used to output the converted standard address text to the input method application through the output layer and display it through the input method application.
[0176] The address text processing device of this embodiment is used to implement the methods of the corresponding address text processing devices in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. Example
[0177] Reference Figure 7 , shows a schematic diagram of the structure of an electronic device according to Embodiment 7 of the present application. The specific embodiments of the present application do not limit the specific implementation of the electronic device.
[0178] like Figure 7As shown, the electronic device may include: a processor (processor) 702 , a communication interface (Communications Interface) 704 , a memory (memory) 706 , and a communication bus 708 .
[0179] in:
[0180] The processor 702 , the communication interface 704 , and the memory 706 communicate with each other via a communication bus 708 .
[0181] The communication interface 704 is used to communicate with other electronic devices such as terminal devices or servers.
[0182] The processor 702 is used to execute the program 710, and specifically can execute the relevant steps in the above-mentioned method embodiment for downloading an application.
[0183] Specifically, the program 710 may include program codes, which include computer operation instructions.
[0184] The processor 702 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0185] The memory 706 is used to store the program 710. The memory 706 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0186] Program 710 can be specifically used to enable the processor 702 to perform the following operations: use the obstruction identifier to complete the administrative division information of the address text to be processed to generate a completed address text; input the completed address text into an address text processing device, and process the completed address text through the address text processing device; obtain the standard address text output by the address text processing device, wherein in the standard address text, the text identified by the obstruction identifier is replaced with standard administrative division information.
[0187] In an optional implementation, the program 710 may also be used to enable the processor 702 to perform standard division training on the address text processing device before using the occluded identifier to complete the administrative division information of the address text to be processed.
[0188] In an optional embodiment, the program 710 is also used to enable the processor 702 to perform a first masking operation on at least part of the address words of the first address text for training, generate pre-training samples based on the first masking result, and use the pre-training samples to pre-train the address text processing device; perform a second masking operation on part of the administrative division data of the second address text for training, and generate target task training samples based on the second masking result; use the target task training samples to perform target training on the pre-trained address text processing device to generate standard administrative division information through the trained address text processing device; wherein, during the target training process, the training parameters of the first N hidden layers in the address text processing device remain unchanged as obtained through the pre-training, and the training parameters of other hidden layers in the address text processing device are trained, and N is a positive integer greater than or equal to 1.
[0189] In an optional embodiment, the program 710 is also used to enable the processor 702 to replace some address words in the address words indicating administrative division data in the second address text with an occlusion identifier; use the second address text replaced with the occlusion identifier as the second occlusion result, and generate a target task training sample based on the second occlusion result.
[0190] In an optional embodiment, the program 710 is also used to enable the processor 702 to generate a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier through the input node of the address text processing device according to each address word and the corresponding position information, the occlusion identifier and the corresponding position information in the target task training sample; distribute the first input word vector or the second input word vector generated by the input node to the hidden layer and output layer of different processing units, so that each processing unit processes the first input word vector and the second input word vector; determine the occlusion identifier according to the processing results of each processing unit. corresponding standard administrative division prediction information; adjusting the training parameters of the other hidden layers in the address text processing device except the first N hidden layers according to the difference between the standard administrative division prediction information and the standard administrative division annotation information corresponding to the obscured address words; returning to the input node of the address text processing device, and continuing to execute the step of generating a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier according to each address word and the corresponding position information, the occlusion identifier and the corresponding position information in the target task training sample, so as to continue to train the address text processing device after adjusting the training parameters until the training termination condition is met.
[0191] In an optional embodiment, the program 710 is also used to enable the processor 702 to obtain a processing result of a processing unit where the input node receiving the second input word vector is located; input the obtained processing result into a classifier of an output node of the processing unit where the processing unit is located, and obtain a classification result of the classifier, and determine the standard administrative division prediction information corresponding to the occlusion identifier based on the classification result and the administrative division dictionary.
[0192] In an optional embodiment, the address text to be processed is a call to receive an input method application, and the address text to be processed sent by the input method application is received through the input layer. The program 710 is also used to enable the processor 702 to output the converted standard address text to the input method application through the output layer and display it through the input method application.
[0193] The specific implementation of each step in program 710 can refer to the corresponding description of the corresponding steps and units in the above-mentioned address text processing method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, which will not be repeated here.
[0194] Through the electronic device of this embodiment, a pre-training sample for pre-training the address text processing device is obtained by performing a first masking operation on the first address text, and then the address text processing device is pre-trained. In this way, the difficulty of obtaining pre-training samples is reduced while ensuring that the address text processing device can learn better training parameters. In the target training process, a second address text corresponding to the target task is used to perform a second masking operation, and a target task training sample is generated according to the second masking result. When the address text processing device is trained using the target task training sample, only the training parameters of the hidden layers in the address text processing device except the first N hidden layers are trained, which improves the training efficiency, and a small amount of target task training samples can be used to achieve a good training effect, reducing the difficulty of target task training. The trained address text processing device can complete the missing administrative division data in the address text and standardize the existing administrative division data therein, solving the problem that the standardized processing method in the prior art needs to pre-formulate more complex matching rules, resulting in the difficulty of matching rule maintenance and requiring more manual intervention, thereby improving the processing efficiency.
[0195] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.
[0196] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the address text processing method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the address text processing method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the address text processing method shown herein.
[0197] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.
[0198] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.
Claims
1. An address text processing device, characterized in that: include: An input layer, the input layer comprising a plurality of input nodes, the input layer being used to receive an address text and encode the address text; A hidden layer, wherein the hidden layer includes a plurality of layers, each layer includes a plurality of hidden layer nodes, the hidden layer is communicatively connected with the input layer, and processes the coded information output by the input layer; An output layer, the output layer is connected to the hidden layer for communication and obtains the processed address text according to the output of the hidden layer; There is an omnidirectional communication connection between the input layer and the hidden layer, and an omnidirectional communication connection between the hidden layer nodes of two adjacent hidden layers; the hidden layer nodes are used to process the context information contained in the address text in the process of processing the encoded information; Wherein, the output layer includes a plurality of output nodes; the first M output nodes of the device are used to output standard administrative division information, where M is a positive integer greater than 1; Wherein, the address text processing device is obtained through standard division training, wherein the standard division training includes pre-training and target training, the pre-training is used to indicate the use of pre-training samples to train the address text processing device, and the target training is used to indicate the use of target task training samples to train the pre-trained address text processing device, the pre-training samples are address texts that obscure at least part of the address words, and the target training samples are address texts that obscure part of the administrative division data.
2. The device according to claim 1, characterized in that Each of the hidden layer nodes comprises: A multi-head self-attention layer, the multi-head self-attention layer is used to extract context information from the encoded information output by the input node corresponding to the current hidden layer node in the input layer and the encoded information output by other input nodes in the input layer; A feedforward network layer, the feedforward network layer is used to receive and process the context information output by the corresponding multi-head self-attention layer.
3. The device according to claim 2, characterized in that Each of the hidden layer nodes further includes a first normalization layer and / or a second normalization layer; The first normalization layer is communicatively connected between the multi-head self-attention layer and the feedforward network layer, normalizes the context information output by the multi-head self-attention layer, and transmits the normalization result to the feedforward network layer; The second normalization layer is communicatively connected to the feedforward network layer and performs normalization processing on the data output by the feedforward network layer.
4. The device according to claim 1, characterized in that The input layer includes a word vector encoding layer and a position information encoding layer; The word vector encoding layer is used to encode the words in the address text into corresponding word vectors; The position information encoding layer is used to encode the word vector into an input word vector according to the position information of the word in the address text, and input the input word vector into the hidden layer as the encoding information.
5. The device according to claim 4, characterized in that The words in the address text include address words and shielding identifiers, or the words in the address text include address words.
6. The device according to claim 1, characterized in that The address text processing device receives a call from an input method application, receives the address text sent by the input method application through the input layer, and after being processed by the input layer and the hidden layer, outputs the converted standard address text to the input method application through the output layer and displays it through the input method application.
7. A method for processing address text, characterized in that: include: The blocked identifier is used to complete the administrative division information of the address text to be processed, and the completed address text is generated; Inputting the padded address text into the address text processing device according to any one of claims 1 to 6, and processing the padded address text by the address text processing device; The standard address text output by the address text processing device is obtained, wherein in the standard address text, the text identified by the obstruction identifier is replaced with standard administrative division information.
8. The method according to claim 7, characterized in that Before using the obstruction identifier to complete the administrative division information of the address text to be processed, the method further includes: The address text processing device is trained on standard divisions.
9. The method according to claim 8, characterized in that The performing standard division training on the address text processing device comprises: Performing a first masking operation on at least part of the address words of the first address text for training, generating a pre-training sample according to the first masking result, and using the pre-training sample to pre-train the address text processing device; Performing a second masking operation on part of the administrative division data of the second address text for training, and generating a target task training sample according to the second masking result; Using the target task training sample to perform target training on the pre-trained address text processing device, so as to generate standard administrative division information through the trained address text processing device; Among them, in the target training process, the training parameters of the first N hidden layers in the address text processing device remain unchanged as the parameters obtained through the pre-training, and the training parameters of other hidden layers in the address text processing device are trained, and N is a positive integer greater than or equal to 1.
10. The method according to claim 9, characterized in that The performing standard division training on the address text processing device comprises: Using a blocking identifier to replace part of the address words in the second address text indicating the administrative division data; The second address text replaced with the occlusion identifier is used as the second occlusion result, and a target task training sample is generated according to the second occlusion result.
11. The method according to claim 9, characterized in that The standard division training of the address text processing device comprises: Generate, through an input node of the address text processing device, a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier according to each address word and corresponding position information, an occlusion identifier and corresponding position information in the target task training sample; Distribute the first input word vector or the second input word vector generated by the input node to the hidden layer and the output layer of different processing units, so that each processing unit processes the first input word vector and the second input word vector; Determining, according to the processing results of each processing unit, standard administrative division prediction information corresponding to the occlusion identifier; According to the difference between the standard administrative division prediction information and the standard administrative division annotation information corresponding to the blocked address word, adjusting the training parameters of other hidden layers except the first N hidden layers in the address text processing device; Returning to the input node of the address text processing device, the step of generating a first input word vector corresponding to each address word and a second input word vector corresponding to the occlusion identifier according to each address word and the corresponding position information, the occlusion identifier and the corresponding position information in the target task training sample continues to be executed, so as to continue training the address text processing device after adjusting the training parameters until the training termination condition is met.
12. The method according to claim 11, characterized in that The standard division training of the address text processing device comprises: Obtaining a processing result of a processing unit where an input node receiving the second input word vector is located; The obtained processing result is input into the classifier of the output node of the processing unit, and the classification result of the classifier is obtained, and the standard administrative division prediction information corresponding to the occlusion identifier is determined according to the classification result and the administrative division dictionary.
13. The method according to claim 8, characterized in that The address text to be processed is a call to receive an input method application, and the address text to be processed sent by the input method application is received through the input layer. The method further includes: The converted standard address text is output to the input method application through the output layer and displayed by the input method application.
14. An address text processing device, characterized in that: include: A placeholder completion module is used to use the mask identifier to complete the administrative division of the address text to be processed and generate the completed address text; A processing module, used for inputting the padded address text into the address text processing device according to any one of claims 1 to 6, and processing the padded address text by the address text processing device; The standard address output module is used to obtain the standard address text output by the address text processing device, wherein in the standard address text, the text identified by the obstruction identifier is replaced by standard administrative division information.
15. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the address text processing method according to any one of claims 7 to 13 is implemented.
Citation Information
Patent Citations
Chinese address word segmentation method based on transformer and CRF
CN110222337A
